Thursday, July 31, 2008

Learning LINQ to SQL (Part 2)

In my last post, I had worked through a few section of the white paper "LINQ to SQL: .NET Language-Integrated Query for Relational Data" and had gotten a working LINQ program going.

Everything was going just fine... until now. At the end of the Defining Relationships section, the article leaves me kind'a hanging. It says "there is a tool (described later) that can be used to generate all the necessary definitions..." So, I've got to go back to the TOC to see what this "special tool" is for generating the Entities.

I jump to "The Entity Class Generator Tool" section and start reading. My first order of business is to find the program called SqlMetal. I launch a COMMAND window and type "sqlmetal" and receive the expected "'sqlmetal' is not recognized as an internal or external command." error. Hey, it was worth a try! So, I next use the old stand-by "cd \" then "dir /s sqlmetal.*". Yup, there it is under the Microsoft SDKs (C:\Program Files\Microsoft SDKs\WIndows\v6.0A\bin).

So, I add the path of the tool to my User PATH variable. I change directories to my project directory and run the command...

SqlMetal /server:.\SQLExpress /database:Northwind /pluralize /namespace:nwind /code:Northwind.cs

SqlMetal reports an error...

Error : Cannot open database "Northwind" requested by the login. The login failed. Login failed for user XALNIX-DEVBOX\Les

To solve this problem, I launch SQL Server Management Studio Express and see why the database won't log me in given my program is working and accessing the database. Once in Management Studio, I note that there is no Northwind database, of course. My program accesses the database by filename. So I attach the Northwind database file and change the database name from the filename to Northwind. Now the SqlMetal runs and produces a file called Northwind.cs.

By the way, if you need the SQL Server Management Studio Express, just follow the link to Microsoft's download page.

Now I can get back to the Learning some LINQ. Before I made the excursion into SqlMetal domain, I was about to enter the section, Querying Across Relationships. Getting back on track, I load the new Northwind.cs into my project. Next, I lift the code snippet from the article and plug it into a new Button2 click handler. Since the snippet from the article does not contain the DataContext, I add a line to create one. And since, the snippet does not have an iteration to look at the results, I use the earlier iterator (with a few tweaks). Now my code looks something like this...


private void button2_Click(object sender, EventArgs e)
{
Northwind db = new Northwind(@"c:\LinqSamples\data\northwnd.mdf");
var q =
from c in db.Customers
from o in c.Orders
where c.City == "London"
select new { c, o };
foreach (var cust in q)
Console.WriteLine("Customer id = {0}, City = {1}, OrderID {2}", cust.c.CustomerID, cust.c.City, cust.o.OrderID);
}
...

[Table(Name = "Customers")]
public class Customer
{
[Column(IsPrimaryKey = true)]
public string CustomerID;
[Column]
public string City;

private EntitySet _Orders;
[Association(Storage = "_Orders", OtherKey = "CustomerID")]
public EntitySet Orders
{
get { return this._Orders; }
set { this._Orders.Assign(value); }
}
}
[Table(Name = "Orders")]
public class Order
{
[Column(IsPrimaryKey = true)]
public int OrderID;
[Column]
public string CustomerID;
private EntityRef _Customer;
[Association(Storage = "_Customer", ThisKey = "CustomerID")]
public Customer Customer
{
get { return this._Customer.Entity; }
set { this._Customer.Entity = value; }
}
}
public partial class Northwind : DataContext
{
public Table Customers;
public Table Orders;
public Northwind(string connection)
: base(connection)
{
}
}


The code would not run though, because SQL Server could not log me into the database. I had attached the Northwnd.mdf in order to get SqlMetal to run. If I were to take the time to figure out the new connection string, I could just change it in my code. But, I cheat and just detach the database. But WATCH OUT! If you create a SQL login for yourself and set the default database to Northwind, and then detach Northwind, you have locked yourself out of the database (if I sound like I have experience with this, [blush] I do).

With the code back to running, I experiment a little. The new query above lets me now get at the members of the Orders with the member "cust.o". The "cust" variable is an anonymous object of type { Customer c, Order o }. So, "intellisense" works great showing the members of the Order class as I type. But, I've just spent a lot of time figuring out SqlMetal, I should be using the Northwind.cs definition it created.

To switch to the auto-generated Entities, I open the Northwind.cs file and change the namespace from "nwind" to LingPOC (to be compatible with my project). I use the handy refactor tool to change all occurances of "nwind". I really don't expect there to be any other occurances, but the tool is very useful in that it reports errors about my existing definitions for Customer and Order not being marked partial. That's okay because I just comment out my three classes (Northwind, Customer, Order). After all that work, gone in a moment.

I run again, everything is still working, and the intellisense now has more information to report about Order and Customer.

The final section of the Quick Tour gets into updating the database. In Modifying and Saving Entities, I note a few points. First, there is a nice little call on the Customers object called "Single()" which gets a single Customer object. The previous examples get a list of objects. So, even if the select statement narrows the results to one row, I would have to use a "foreach" statement to get at the row. The "Single()" statement reduces the code and complexity. Also, I note that creating a new Order appears very database independent, nice. Last, I note that the update to the database is performed by the "SubmitChanges()" member of the DataContext. Hmm, if I do a query for a customer and a query for an order on the same context, it appears separate changes to each cannot be separately updated to the database. I'll have to check this out.

Yes, it seems so. A single DataContext will update multiple separate selections. But, if I use separate DataContexts, I can independantly control the updates. There is a note that this behavior can be overridden. Anyway, that's enough for now. In a future post, I will embark upon the next section of the article, Queries in Depth. Here's my updated code. You'll need to generate your own Northwind.cs. And if you use this code, be sure to add the buttons to your form and link up the button click handlers. Enjoy...


using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.Data.Linq;
using System.Data.Linq.Mapping;

namespace LinqPOC
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}

private void button1_Click(object sender, EventArgs e)
{
Northwind db = new Northwind(@"c:\LinqSamples\data\northwnd.mdf");
// Query for customers from London
var q =
from c in db.Customers
where c.City == "London"
select c;
foreach (var cust in q)
Console.WriteLine("id = {0}, City = {1}", cust.CustomerID, cust.City);
}

private void button2_Click(object sender, EventArgs e)
{
Northwind db = new Northwind(@"c:\LinqSamples\data\northwnd.mdf");
var q =
from c in db.Customers
from o in c.Orders
where c.City == "London"
select new { c, o };
foreach (var cust in q)
Console.WriteLine("Customer id = {0}, City = {1}, OrderID {2}", cust.c.CustomerID, cust.c.City, cust.o.OrderID);
}

private void button3_Click(object sender, EventArgs e)
{
Northwind db = new Northwind(@"c:\LinqSamples\data\northwnd.mdf");
var cust = db.Customers.Single( c => c.CustomerID == "ALFKI");
var ord = db.Orders.Single( o => o.OrderID == 10269);// customer WHITC

ord.OrderDate = DateTime.Now;

Order order = new Order();
order.OrderDate = DateTime.Now;
cust.Orders.Add(order);

db.SubmitChanges();
}

private void button4_Click(object sender, EventArgs e)
{
Northwind db = new Northwind(@"c:\LinqSamples\data\northwnd.mdf");
Northwind db2 = new Northwind(@"c:\LinqSamples\data\northwnd.mdf");
var cust = db.Customers.Single(c => c.CustomerID == "ALFKI");
var ord = db2.Orders.Single(o => o.OrderID == 10269);// customer WHITC

ord.OrderDate = DateTime.Now;

Order order = new Order();
order.OrderDate = DateTime.Now;
cust.Orders.Add(order);

db.SubmitChanges();
db2.SubmitChanges();
}
}
}



(To be continued...)

Learning LINQ to SQL

I've decided to spend a little time these next few evenings coming up to speed on LINQ (Language-Integrated Query) and writing about my experience with it. Let's see what sorts of problems I run into. I'm using as reference the 119 page LINQ to SQL: .NET Language-Integrated Query for Relational Data in order to jump start my learning. I hope that my SQL and C# experience will give me a head start and keep me moving. But, I have done nothing yet with the Northwind sample database (the database used in the white paper) in any of my projects or tests. I am not even sure where to get it yet. Ha, ha, Noobie on the loose, let me at it...

Let's Create some Entity Classes. The white paper starts out with creating entity classes. Granted, I'm starting in the "Quick Tour" section of the white paper, so I expect to find some detail ommisions. I'll take my chances. So, I jump right into the declaration of the Customer class.

[Table(Name="Customers")]
public class Customer
{
public string CustomerID;
public string City;
}

Well, by looking at it, I can see I am probably missing a few things already. I compile and sure enough, the errors tell me I need to reference something. My hunch is I need something like "using System.Data.Linq;", but since intellisense doesn't show it, I look under References instead and find "System.Data.Linq". I add it and now the intellisense is happy. I attempt to compile again.

The next stumbling block is the error that System.Data.Linq.Table<TEntity> requires a type argument. Hmm, let's hit MSDN and search for TableAttribute. Why not "Table<TEntity>" you ask? That's because the only "Table" reference in my program is [Table...] and I know this syntax is shorthand for [TableAttribute...]. Now, I could go looking for "Table<TEntity>", but the real problem is TableAttribute is not being found in one of my references, so the compiler is finding something else. I'll figure out the something else later. Right now, I'm on a mission. The search brings up "TableAttribute Class (System.Data.Linq.Mapping)". As I thought, I was missing another reference. I add the "using System.Data.Linq.Mapping;" and I'm back on track.

After adding column attributes and checking out my compile again, I next come to "The DataContext". This part of the white paper calls for "northwnd.mdf". Well, I've never used the Northwinds example database, so I have no clue where to find the file. I start searching the hard drive, and of course, I hit the search engines again searching for "northwnd.mdf sample file". Look's like someone else was having trouble finding it, too. The search rendered "Can't find Northwnd.mdf example database", so I read up. But my search of the hard drive also turns up a copy in the LinqSamples/Data directory. So, I use that (actually, I extract it from the CSharpSamples.zip file in c:\<install-location>\Samples\1033 ).

With database file in place and all code adjusted for the correct path, I compile and run. The next error I hit is an exception when I interate through my query results. The code up to this point looks like this...


private void button1_Click(object sender, EventArgs e)
{
// DataContext takes a connection string
DataContext db = new DataContext(@"c:\LinqSamples\data\northwnd.mdf");
// Get a typed table to run queries
Table Customers = db.GetTable();
// Query for customers from London
var q =
from c in Customers
where c.City == "London"
select c;
foreach (var cust in q)
Console.WriteLine("id = {0}, City = {1}", cust.CustomerID, cust.City);
}
...

[Table(Name="Customers")]
public class Customer
{
[Column(IsPrimaryKey=true)]
public string CustomerID;
[Column]
public string City;
}


The exception is occuring when I step through the code and land on "q" in...

foreach(var cust in q)

... That clues me in that database connections are not made until later than I expect, and some of this new LINQ syntax does not run immediately. Interesting. The exception says effectively that I don't have a SqlServer running, and upon checking my configuration, I find my SqlServer Express edition is stopped and set to start up manually. So, I turn it on and run my first attempt at a LINQ program. My output is ...

id = AROUT, City = London
id = BSBEV, City = London
id = CONSH, City = London
id = EASTC, City = London
id = NORTS, City = London
id = SEVES, City = London

Wow, that wasn't too painful. I have a working LINQ program. I recognize the white paper was not intended to be a tutorial, so I'm pleased with how few are the problems I've run into thus far. Maybe I should have started with a tutorial, but the level of detail here is perfect for someone like myself who is after a really rapid ramp-up, and who has enough experience to solve some of the basic problems on his own.

The DataContext is a little too loose for the author of the article, so I follow along by deriving the Northwind class from the DataContext and updating my code.


public partial class Northwind : DataContext
{
public Table Customers;
public Table Orders;
public Northwind(string connection)
: base(connection) {}
}

...

private void button1_Click(object sender, EventArgs e)
{
Northwind db = new Northwind(@"c:\LinqSamples\data\northwnd.mdf");
// Query for customers from London
var q =
from c in db.Customers
where c.City == "London"
select c;
foreach (var cust in q)
Console.WriteLine("id = {0}, City = {1}", cust.CustomerID, cust.City);
}
...

[Table(Name="Customers")]
public class Customer
{
[Column(IsPrimaryKey=true)]
public string CustomerID;
[Column]
public string City;
}
[Table(Name = "Orders")]
public class Order
{
}


So, with this declaration (and having to define the Order class on my own), I can now do without the DataContext and the GetTable<Customer>() call. I compile again and run it successfully. So, I proceed to "Defining Relationships".

Here the example code has an error. It applies the ColumnAttribute with an invalid parameter "Id". I believe the correct parameter to be "IsPrimaryKey", so I use that instead. I also find that here is where the Order class is defined, so I use this definition and fix the ColumnAttribute on OrderID as well (change "Id" to "IsPrimaryKey"). It all compiles, but I am concerned that neither "_Orders" nor "_Customer" are ever instantiated. Well, I don't know how "Storage=" works, so I look at it once more cautiously before moving on.

(to be continued...)

Wednesday, July 30, 2008

Timers are a Changin' (Part 3)

In my last couple of articles, we saw the System.Windows.Forms.Timer and the System.Timers.Timer. Today, I shall look at the 3rd form of Timer in the .NET library, the System.Threading.Timer. I refer you again to the that very good article I mentioned a while back, Comparing the Timer Classes in the .NET Framework Class Library by Alex Calvo. Take a moment to read his section on System.Threading.Timer then come back and we will jump right into some code...


using System.Threading;
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
timer1 = new System.Threading.Timer(new TimerCallback(timer_elapsed), null, 2000, 1000);
}

System.Threading.Timer timer1;
private void timer_elapsed(object sender)
{
Console.WriteLine(DateTime.Now.ToString());
TimerEvent(DateTime.Now.ToString(), Thread.CurrentThread.GetHashCode().ToString());
}

private delegate void TimerEventDelegate(string label, string name);
private void TimerEvent(string label, string name)
{
if (label1.InvokeRequired)
{
BeginInvoke(new TimerEventDelegate(TimerEvent), new object[] { label, name });
}
else
{
label1.Text = "Thread:" + name + " - Time:" + label;
}
}
}



To use this code above, create a new project named TimerExample3 and drop a label on the form. Then copy the code into your form1.cs replacing everything inside of the Namespace. It's ready to run!

The extra features gained with this timer are the ability to tell it to start at a future time, and you can pass it a state object. But there is another feature that is not so obvious. That is, the timer actually runs on the thread pool. If the timer_elapsed handler were to get hung in a sleep or something, that's okay because the next timer event will occur on time and fire the timer_elapsed again on a different thread. But, watch out! If you do not program for re-entrancy, you'll likely have problems.

At the moment, I have no guards for re-entrancy because the Console.WriteLine and the BeginInvoke should finish well within 1 second interval. So then, let's look at the behavior. First, notice that it takes 2 seconds before anything happens. That's in accordance with the 2000 millisecond startup time I gave the timer. Then you see the label updated with the thread number and time once every second along with console output. This is roughly the same as the last projects, only it took some very different code to get there.

Now, lets add a modal dialog to tie up the GUI thread and a button to tell the GUI to sleep as we did before. Here are the handlers, just drop two buttons on the form and wire up the handlers.


private void button1_Click(object sender, EventArgs e)
{
FolderBrowserDialog f = new FolderBrowserDialog();
f.ShowDialog();
}
private void button2_Click(object sender, EventArgs e)
{
Thread.Sleep(5000);
}


Run the application and experiment with the buttons. The console output does not miss a beat regardless of which button you click. The form continues to update the label with the time while the modal dialog is visible, but not when clicking the button to sleep. So far, so good. No real difference from our last timer.

Next, modify the handler by adding a Sleep() call after the TimerEvent() call. This will simulate some long running timer event processing for the sake of making the event handler exceed the programmed interval. Then run and look at the thread number and time.


private void timer_elapsed(object sender)
{
Console.WriteLine(DateTime.Now.ToString());
TimerEvent(DateTime.Now.ToString(), Thread.CurrentThread.GetHashCode().ToString());
Thread.Sleep(2000);
}


Sleep(2000) ties up the thread running timer_elapsed() callback. So, the Timer simply allocates another thread to handle the next timer interval. You can see this in the thread number in the form label, there are different thread numbers for each update, probably alternating between two values as my version does. If you recall then, the System.Windows.Forms.Timer runs in the same thread as the form. The System.Timers.Timer runs in a thread different than the GUI and has to be synchronized to the GUI of choice. Now we see that the System.Threading.Timer runs in a separate thread allocated from the thread pool.

This got me to thinking. Is the System.Timers.Timer also allocated from the thread pool? With a few quick modifications to my TimerExample2 application, I found the answer is yes!

I thought the System.Timers.Timer it would be on its own dedicated thread. Finding that it's not, we need to realize that we must deal with re-entrancy in both System.Timers.Timer and System.Threading.Timer. This was not mentioned or wasn't clear in the article I referenced. The good news is that you can treat re-entrancy the same way for both handlers.

So, the only real new features of the System.Threading.Timer are the delayed start and being able to pass in a state variable. So I will be revisiting my last article to correct any dis-information I may have provided about single-threadedness. But, now I wonder, will the System.Timers.Timer "elapsed" handler provides useful information in the "object sender" parameter? Does this serve as nearly a state variable? Maybe, I will post again soon on these and other questions I find.

For now, here's the code for TimerExample3. I figure I should just let you have what we've got until more research has been done.


using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;

namespace TimerExample3
{
using System.Threading;
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
timer1 = new System.Threading.Timer(new TimerCallback(timer_elapsed), null, 5000, 1000);
}

System.Threading.Timer timer1;
private void timer_elapsed(object sender)
{
Console.WriteLine(DateTime.Now.ToString());
TimerEvent(DateTime.Now.ToString(), Thread.CurrentThread.GetHashCode().ToString());
}

private delegate void TimerEventDelegate(string label, string name);
private void TimerEvent(string label, string name)
{
if (label1.InvokeRequired)
{
BeginInvoke(new TimerEventDelegate(TimerEvent), new object[] { label, name });
}
else
{
label1.Text = "Thread:" + name + " - Time:" + label;
}
}
private void button1_Click(object sender, EventArgs e)
{
FolderBrowserDialog f = new FolderBrowserDialog();
f.ShowDialog();
}
private void button2_Click(object sender, EventArgs e)
{
Thread.Sleep(5000);
}
}
}



... and here is my updated version of TimerExamples2, notice that I added a Button6 ...


using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.IO;
using System.Threading;

namespace TimerExample2
{
public partial class Form1 : Form
{
System.Timers.Timer timer1;
System.Timers.Timer timer2;
public Form1()
{
InitializeComponent();
timer1 = new System.Timers.Timer(1000);
timer1.SynchronizingObject = this;
timer1.Elapsed += new System.Timers.ElapsedEventHandler(timer1_Elapsed);
timer1.Start();
}

private string theText(string signal)
{
string name = "Thread: " + Thread.CurrentThread.GetHashCode().ToString();
string stime = "Time: " + DateTime.Now.ToString("hh:mm:ss");
string s = name + " - " + stime + " - SignalTime: " + signal;
return s;
}
void timer1_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
{

Console.WriteLine("Timer1 {0}",theText(e.SignalTime.ToString()));
label1.Text = theText(e.SignalTime.ToString());
}
void timer2_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
{
string s = theText(e.SignalTime.ToString());
Console.WriteLine("Timer2 {0}",s);
label2.Invoke(new MethodInvoker(() => label2.Text = s));
}
void timer2_ElapsedAsynch(object sender, System.Timers.ElapsedEventArgs e)
{
string s = theText(e.SignalTime.ToString());
Console.WriteLine("Timer2 {0}", s);
label2.BeginInvoke(new MethodInvoker(() => label2.Text = s));
}
void timer2_ElapsedAsynchSleep(object sender, System.Timers.ElapsedEventArgs e)
{
string s = theText(e.SignalTime.ToString());
Console.WriteLine("Timer2 {0}", s);
label2.BeginInvoke(new MethodInvoker(() => label2.Text = s));
Thread.Sleep(2000);
}

private void button1_Click(object sender, EventArgs e)
{
FolderBrowserDialog fb = new FolderBrowserDialog();
fb.ShowDialog();
}

private void button2_Click(object sender, EventArgs e)
{
Form2 frm = new Form2();
frm.ShowDialog();
}

private void button3_Click(object sender, EventArgs e)
{
Thread.Sleep(5000);
}

private void button4_Click(object sender, EventArgs e)
{
if (timer2 != null)
{
timer2.Stop();
timer2.Dispose();
timer2 = null;
}
else
{
timer2 = new System.Timers.Timer(1000);
timer2.Elapsed += new System.Timers.ElapsedEventHandler(timer2_Elapsed);
timer2.Start();
}
}

private void button5_Click(object sender, EventArgs e)
{
if (timer2 != null)
{
timer2.Stop();
timer2.Dispose();
timer2 = null;
}
else
{
timer2 = new System.Timers.Timer(1000);
timer2.Elapsed += new System.Timers.ElapsedEventHandler(timer2_ElapsedAsynch);
timer2.Start();
}
}

private void button6_Click(object sender, EventArgs e)
{
if (timer2 != null)
{
timer2.Stop();
timer2.Dispose();
timer2 = null;
}
else
{
timer2 = new System.Timers.Timer(1000);
timer2.Elapsed += new System.Timers.ElapsedEventHandler(timer2_ElapsedAsynchSleep);
timer2.Start();
}
}
}
}

Friday, July 25, 2008

Timers are a Changin' (Part 2)

Today, I take a look at System.Timers.Timer. I suggested an article in my last discussion of timers. The article was written back in 2004, and it says that both the System.Windows.Forms.Timer and the System.Timers.Timer are found in the IDE Toolbox, one on the Windows Forms tab and the other on the Components tab. Well in my stock installation of Visual Studio 2008, both of those timers are System.Windows.Forms.Timer components. This may be due to some install problem I didn't know I had, or it may be the article is correct for Visual Studio .Net 2003 but not 2008.

That's okay because you can just add System.Timers.Timer to your project the old fashioned way. I simply start a new Windows Forms project and name it TimerExample2. After dropping a label on my form, I add the following code to my class file (I include a full listing at the end).


System.Timers.Timer timer1; //added
public Form1()
{
InitializeComponent();
// added the following
timer1 = new System.Timers.Timer(1000);
timer1.SynchronizingObject = this;
timer1.Elapsed += new System.Timers.ElapsedEventHandler(timer1_Elapsed);
timer1.Start();
}
void timer1_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
{
label1.Text = DateTime.Now.ToString("hh:mm:ss");
Console.WriteLine(e.SignalTime.ToString());
}


When I compile and run this application, I have a simple clock ticking away the seconds. Behaviorally speaking, its not much different from the last timer project. By keeping things simple, I can take a closer look at the differences. Besides the different constructor and handler, the thing that most stands out is the "SynchronizingObject" property. This property clues us in to one of the most fundamental differences between the standard Timer and the System.Timers.Timer. The System.Timers.Timer will actually run on its own dedicated thread. If I ran without setting the SynchronizingObject to "this" (my Form1), the timer would not have direct to my main GUI label1. Try it yourself, comment out the line that sets the SynchronizingObject and try running the application. It will throw an exception when it reaches the line that sets label1.Text. The SynchronizingObject in effect determines the thread on which the event handler will run, the timer still runs in a separate thread, but its event handler is run on the thread of the form you choose (in my case I chose Form1). Or, as I said, if you don't set the SynchronizingObject, the timer's event runs on its own thread and any interaction with the GUI will require Invoke type calls.

So let's test what I just said with some code. Let's go ahead and add some buttons and code like we did in our last discussion. I have a Button1 to launch a modal dialog box, Button2 to launch a modal form (Form2), and Button3 to "sleep" for 5 seconds. The modal operations stop the main event loop and start a sub-event loop, while the "Sleep" pauses the thread and all its event loops. I've wired these all up using the IDE, and I created a Form2 to be called. The handlers are below.


private void button1_Click(object sender, EventArgs e)
{
FolderBrowserDialog fb = new FolderBrowserDialog();
fb.ShowDialog();
}

private void button2_Click(object sender, EventArgs e)
{
Form2 frm = new Form2();
frm.ShowDialog();
}

private void button3_Click(object sender, EventArgs e)
{
Thread.Sleep(5000);
}


When you run the application and click the first button, you will notice the timer continues to run even though we have a modal dialog. But, that's no surprise because that's what the last Timer we looked at did. We determined last time that the modal dialog's event loop was picking up the events and dispatching the parent's timer events though the other events were being eaten while the modal dialog was open. Similarly, clicking on the second button shows a form with the ShowDialog() call, a modal call. The timer still ticks away for the same reasons. And, when you click the third button, the clock stops ticking for 5 seconds. It doesn't appear to be any different until you take a moment to review the console output. When the Sleep returns, there are actually 5 timer events processed immediately following. Herein lies the difference. During the sleep the System.Timers.Timer continued to operate and post tick events. The last timer we looked at, System.Windows.Forms.Timer, does not post these events, its ticks are lost. By operating in its own thread, System.Timers.Timer is not effected by the GUI thread's call to Sleep().

To further demonstrate the threaded-ness of this Timer, let's look as some more code. I add another label (label2) and another Timer (timer2) to Form1. Then I add a handler for timer2 and a Button4 which instantiates and starts the timer, and some additional information to the Console.WriteLine to distinguish the two event handlers' output. The added code basically looks like this...


System.Timers.Timer timer2;
void timer2_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
{
string s = DateTime.Now.ToString("hh:mm:ss");
label2.Invoke(new MethodInvoker(() => label2.Text = s ));
Console.WriteLine("Timer2 {0}",e.SignalTime.ToString());
}

private void button4_Click(object sender, EventArgs e)
{
if (timer2 != null)
{
timer2.Stop();
timer2.Dispose();
timer2 = null;
}
else
{
timer2 = new System.Timers.Timer(1000);
timer2.Elapsed += new System.Timers.ElapsedEventHandler(timer2_Elapsed);
timer2.Start();
}
}


If you run the application now, when you click Button4, the second timer starts ticking away, and when you click it again, it stops. When you click Button3, the main GUI sleeps for 5 seconds and both timers stop, but when the GUI awakes, notice that only timer1 continued to post events. What happened to timer2's events? Timer2 posts its events to the GUI through Invoke(). Invoke() is a blocking/synchronous call, so though timer2 is on its own thread, the thread was stopped waiting for Invoke to return which in turn was waiting for the GUI to service the invoke request, which couldn't happen while the GUI was asleep. So, I add Button5 and another event handler for timer2.


void timer2_ElapsedAsynch(object sender, System.Timers.ElapsedEventArgs e)
{
string s = DateTime.Now.ToString("hh:mm:ss");
label2.BeginInvoke(new MethodInvoker(() => label2.Text = s));
Console.WriteLine("Timer2 {0}", e.SignalTime.ToString());
}
private void button5_Click(object sender, EventArgs e)
{
if (timer2 != null)
{
timer2.Stop();
timer2.Dispose();
timer2 = null;
}
else
{
timer2 = new System.Timers.Timer(1000);
timer2.Elapsed += new System.Timers.ElapsedEventHandler(timer2_ElapsedAsynch);
timer2.Start();
}
}


Now, when you click Button5 to start the timer2 and then click Button3 to put the GUI to sleep, both labels stop updating, but the console output shows that timer2 continues to fire events. When the sleep completes, the timer1 events appear in the output. This is still not quite the same behavior as using the SynchronizingObject which is actually post a message to the GUI to fire the event handler. Our little example fires the event handler on the Timer thread and posts an asynchronous message to Invoke the update on "Text". But we don't need the exact same behavior. In order to make the best use of a timer on a dedicated thread, its best to let the tick event handler process rather than blocking it through Invoke() or by synchronizing it through the GUI event loop. This way, should the GUI sleep (which it should not) or get bogged down with lots of events (which it very well could), the timer on its own thread will still get its event on time (roughly). And if the CPU isn't too consumed, the timer thread should get enough cycles to perform your custom timer operations.

Well, I really wanted to get this article out much sooner than this. But I still would like to see how the System.Timers.Timer works in conjunction with an event loop on a different thread than the main GUI. For example, put Form1 on one thread, Form2 on another thread and then play around with the SynchronizingObject variable and see what happens. But alas, I need to post this and move on to the last Timer in my next article. Maybe I'll come back to these, or you can post your findings.

Anyway, here's the code for the article. You need to ...
1) start a project called TimerExample2,
2) drop a couple of labels on Form1,
3) drop 5 buttons on Form1,
4) add another Form (Form2),
5) replace the code in your form1.cs with the code below, and
6) connect each of your buttons' click handlers to the appropriate handler below


using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.IO;
using System.Threading;

namespace TimerExample2
{
public partial class Form1 : Form
{
System.Timers.Timer timer1;
System.Timers.Timer timer2;
public Form1()
{
InitializeComponent();
timer1 = new System.Timers.Timer(1000);
timer1.SynchronizingObject = this;
timer1.Elapsed += new System.Timers.ElapsedEventHandler(timer1_Elapsed);
timer1.Start();
}

void timer1_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
{
label1.Text = DateTime.Now.ToString("hh:mm:ss");
Console.WriteLine("Timer1 {0}",e.SignalTime.ToString());
}
void timer2_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
{
string s = DateTime.Now.ToString("hh:mm:ss");
label2.Invoke(new MethodInvoker(() => label2.Text = s ));
Console.WriteLine("Timer2 {0}",e.SignalTime.ToString());
}
void timer2_ElapsedAsynch(object sender, System.Timers.ElapsedEventArgs e)
{
string s = DateTime.Now.ToString("hh:mm:ss");
label2.BeginInvoke(new MethodInvoker(() => label2.Text = s));
Console.WriteLine("Timer2 {0}", e.SignalTime.ToString());
}

private void button1_Click(object sender, EventArgs e)
{
FolderBrowserDialog fb = new FolderBrowserDialog();
fb.ShowDialog();
}

private void button2_Click(object sender, EventArgs e)
{
Form2 frm = new Form2();
frm.ShowDialog();
}

private void button3_Click(object sender, EventArgs e)
{
Thread.Sleep(5000);
}

private void button4_Click(object sender, EventArgs e)
{
if (timer2 != null)
{
timer2.Stop();
timer2.Dispose();
timer2 = null;
}
else
{
timer2 = new System.Timers.Timer(1000);
timer2.Elapsed += new System.Timers.ElapsedEventHandler(timer2_Elapsed);
timer2.Start();
}
}

private void button5_Click(object sender, EventArgs e)
{
if (timer2 != null)
{
timer2.Stop();
timer2.Dispose();
timer2 = null;
}
else
{
timer2 = new System.Timers.Timer(1000);
timer2.Elapsed += new System.Timers.ElapsedEventHandler(timer2_ElapsedAsynch);
timer2.Start();
}
}
}
}

Thursday, July 24, 2008

Timers are a Changin' (Part 1)

Sometimes mundane things turn out to be really nice after all. No, I'm not talking about one of your last dates. I'm talking about timers, yes, the Timer classes in C#. These little tools are found everywhere in programming, and it seemed for the longest time, there was only one stock timer to choose from. Well, now there are three.

Before I go any further, let me recommend to you Alex Calvo's MSDN magazine article Timers: Comparing the Timer Classes in the .NET Framework Class Library. Here, you will find some good background for applying timers to your application.

But, in this little series, I want to look at how the timers interact with the event loop. This first article in the series will look at the System.Windows.Form.Timer.

The System.Windows.Form.Timer is supposed to be synchronous with respect to the rest of your Windows Forms app. That means, if you "sleep" or block in some other way, the timer will stop working (while sleeping). However, programmers forget the one caveat. That is, the timer stops working unless "DoEvents" is called (or it's equivalents, let me add). Take a look a the following example. Create a new Forms project. From the toolbox, drop a timer, a button, and a label on the form. Replace the code inside your namespace with the code below. Add a "using System.IO;" to the top. Then link the click and timer events to the appropriate functions below.


public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
timer1.Start();
}

private void button1_Click(object sender, EventArgs e)
{
FolderBrowserDialog f = new FolderBrowserDialog();
f.ShowDialog(this);
}

private void timer1_Tick(object sender, EventArgs e)
{
label1.Text = DateTime.Now.ToString("hh:mm:ss");
}
}


When you run the example, the time is displayed and updated ever 1/10 of a second (the default 100ms tick). And when you click the button, a FolderBrowserDialog is launched, and the time keeps ticking? Doesn't the article we just read say the timer is synchronous with the rest of our Windows Form app. It says it won't preempt our code as long as we don't call DoEvents(). Yet the modal FolderBrowserDialog has blocked all our other events from being processed. What's going on?

When ever things don't behave as you expect, you have an opportunity to learn something. So, I made a few changes to the code above so that I can analyse further. I added a button to launch a modal MessageBox and a button to call System.Threading.Thread.Sleep(). Now I have...


public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
timer1.Start();
}

private void button1_Click(object sender, EventArgs e)
{
FolderBrowserDialog f = new FolderBrowserDialog();
f.ShowDialog(this);
}

private void timer1_Tick(object sender, EventArgs e)
{
label1.Text = DateTime.Now.ToString("hh:mm:ss");
}

private void button2_Click(object sender, EventArgs e)
{
MessageBox.Show("hello");
}

private void button3_Click(object sender, EventArgs e)
{
System.Threading.Thread.Sleep(5000);
}
}


Now, when you click on button2, a MessageBox is displayed, and the timer keeps ticking. But if you click on button3, the timer pauses for 5 seconds. So, some operations will stop the timer and some won't. Sleep() will suspend the timer events. But, FolderBrowserDialog and MessageBox and other file dialogs will not. What operations will suspend the timer events? And which operations are modal but will not interfere with the timer? It's important to know because the operations that interfer with the event loop (like Sleep) will interfer with more than just the timer, they will make your application appear to hang.

To answer this question, run the code again, but while its running with no buttons having been clicked, insert a breakpoint in the timer1_tick callback. When the code breaks at this breakpoint, look at the call stack. You should see some external code, above that the main appliction, above that some more external code, and above that the timer1_tick at the top of the stack. Right click on the external code stack element and select the "show external code" context menu item. Reading the stack you can see that above Main() we have the Application.Run() call and some additional calls. These calls implement the main event loop. A timer tick triggers the callback and you find the debuggable callback and further calls to the timer1_tick callback.

Clear the break point and continue your program. Now click button one so that the FolderBrowserDialog opens. Next, set your breakpoint in timer1_Tick() again. Looking at the stack trace this time, we see external code, above that Main(), above that external code, above that button1_Click(), above that somemore external code, and finally above that timer1_Tick(). We deduce a couple important things based upon this experiment. 1) All the controls, including the timer and the FolderBrowserDialog, run in the same thread. That is why sleeping your thread makes everything stop responding. 2) Modal windows do not block the thread (nor do they call DoEvents), they are windows forms in and of their own right and are not required to know or care that they have a parent window. In fact, if you add another Form to the project (say Form2.cs), place a button on it to close the form, and add a button to Form1 to create and show the form using ShowDialog(), then you will see the same behavior.

But, why then are the timer events getting through to Form1 at all while mouse events and other events are not? If the call stack shows the timer1_Tick() getting called deep in the call stack well away from the parent's event loop. The answer lies in the fact that ShowDialog has its own internal event loop. This event loop receives the timer_Tick event, but since it does not belong to this loop, its dispatcher processes the event on the parent window. Some events are eaten by the dispatcher in order to implement "modality", others are not.

The following has button4 added to show the new form. You would have to add a button to your Form1 and link in the button4_Click() handler. And you would need to implement a simple Form2.cs, mine just has an exit button.


public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
timer1.Start();
}

private void button1_Click(object sender, EventArgs e)
{
FolderBrowserDialog f = new FolderBrowserDialog();
f.ShowDialog(this);
}

private void timer1_Tick(object sender, EventArgs e)
{
label1.Text = DateTime.Now.ToString("hh:mm:ss");
}

private void button2_Click(object sender, EventArgs e)
{
MessageBox.Show(this, "hello");
}

private void button3_Click(object sender, EventArgs e)
{
System.Threading.Thread.Sleep(5000);
}

private void button4_Click(object sender, EventArgs e)
{
Form2 frm = new Form2();
frm.ShowDialog(this);
}
}


We have just shown that opening a form with ShowDialog() puts the forms new event loop on the same thread as the main GUI event loop. Form2 is shown modally, so Form1 stops responding to all events except the timer event. But, what if I start a new thread for the second form and ShowDialog() on the new thread, will the event loop run on the new thread separate from the main GUI thread? Yes, but this is well off topic, maybe I'll discuss that in another article.

Well we learned almost nothing about configuring a timer control. But, you should already know how to do that anyway. I simply hope this discussion gives you a little more insight into how the stock System.Windows.Form.Timer will operate under different situations.

In my next article in this series, I will look at the System.Timers.Timer in more detail.

Friday, July 18, 2008

Why Try? Finally!

Today we talk about the "try/finally" construct. This article should be very simple for most of you with any C# or C++ experience to understand. I've written a sample that you will find at the end of this article to let you see the various affects of the "finally" statement under different scenarios.

But why use "finally" at all? I like to think of "finally" as being to the flow of logic what the "destructor" or "dispose" is to the life of an object. It is a common place to put all the necessary logic for shutting down the logic context. With the ubiquitous "execption" lurking behind every corner, you never know when you will be forced to leave the context, and it's nice to have a place for common cleanup code. You say you could just write the cleanup in the common exception handler. But then you have to repeat the code, line for line, in every other named exception handler as well as the error free path of execution (not to mention, you may just want the exception to go unhandled). The "finally" construct allows you to avoid duplication of logic or insertion of unnecessary logic. One of my favorite examples of the try/finally construct is the use of "Monitor".


Monitor.Enter(lockObject);
try {
// critical section
}
finally {
Monitor.Exit(lockObject);
}


Here, if the Monitor.Enter() is successful, it MUST be followed by a Monitor.Exit() or else everthing could come to a halt. This example has been replaced by the built-in compiler directive "lock(lockObject){...}" but that isn't always the best choice.

Let me repeat. The main usefulness of "finally" is to avoid duplication of cleanup logic whether normal or exceptional. It provides one place to write the logic (and maintain it).

Here is a break-down of the how the this features behaves. First, we start with one of the simplest examples of try/finally.


try {
// do your thing
Console.WriteLine("Example 1: In Try.");
}
finally {
// run some code that must alway follow our "try"
Console.WriteLine("Example 1: In Finally");
}
// stuff to run after the try/finally
Console.WriteLine("Example 1: After the Try/Finally");


The compiler guarantees that the "finally" block of code will run upon the exit of the "try" block, regardless of how the block exist. In Example 1, the try block exits according to the normal flow of code falling out of the block. Example 2 demonstrates some more normal flow, but this time it is through the "return" statement.


try
{
Console.WriteLine("Example 2: Try called");
return;
}
finally
{
Console.WriteLine("Example 2: Finally called");
}
Console.WriteLine("Example 2: More code called");


Here you can see that even though we "return" within the "try", the "finally" is still run. But, the stuff following the "finally" block does not run. That code is effectively unreachable now. But what happens if an exception is thrown in the "try" block. The "finally" should still be called as the next example demonstrates. I have included a "catch" to absorb the exception for the sake of the example.


try
{
Console.WriteLine("Example 3: Try called");
throw new Exception("Example 3: Exception thrown");
}
catch (Exception ex)
{
Console.WriteLine("Example 3: Exception Caught: ErrMsg={0}", ex.Message);
}
finally
{
Console.WriteLine("Example 3: Finally called");
}
Console.WriteLine("Example 3: More code called");


You may have noticed that the last line of code following the finally block executes this time, but note the order. First the "try" executes, then the "catch" executes, then the "finally" and then the code following the "finally" block. Again, "finally" works as advertised. We could also "return" from the "catch" block and "finally" will still run.


try
{
Console.WriteLine("Example 4: Try called");
throw new Exception("Example 4: Exception thrown");
}
catch (Exception ex)
{
Console.WriteLine("Example 4: Exception Caught: ErrMsg={0}", ex.Message);
return;
}
finally
{
Console.WriteLine("Example 4: Finally called");
}
Console.WriteLine("Example 4: More code called");


... and of course, the line following the "finally" block does not execute because of the return. You say, "yeh, but you are catching the exception, what if you don't catch the exception?" I'm glad you asked. This next example makes use of a helper function to demonstrate just that. In the helper function, there is a "try/finally" that throws an exception in the "try". The exception is not caught in that scope and context. I've added a catch higher up in the stack to allow you to prove these examples in a debugger. You could do without the higher catch if you want to compile and run the application outside of debug mode. But, everything is eventually caught, even if it is by the run-time. Regardless, here is the example.


try
{
button5helper();
}
catch (Exception ex)
{
Console.WriteLine("Example 5: Outer catch: ErrMsg={0}",ex.Message);
return;
}
...
private void button5helper()
{
try
{
Console.WriteLine("Example 5: Try called");
throw new Exception("Example 5: Exception thrown");
}
finally
{
Console.WriteLine("Example 5: Finally called");
}
Console.WriteLine("Example 5: More code called");
}


The example will run "try", then "finally", then the "catch" in the outer context. The last line after the "finally" block is not run. But you ask, "what happenens when the "finally" throws an exception?" This next example will show you.


try
{
button6helper();
}
catch (Exception ex)
{
Console.WriteLine("Example 6: Outer catch: ErrMsg={0}",ex.Message);
return;
}
...
private void button6helper()
{
try
{
Console.WriteLine("Example 6: Try called");
throw new Exception("Example 6: Exception thrown");
}
finally
{
Console.WriteLine("Example 6: Finally called");
throw new Exception("Example 6: Exception thrown by finally");
}
Console.WriteLine("Example 6: More code called");
}


Here, the "try" is called wich throws an exception, which results in the finally being called which throws an exception, followed by the "catch" in the outer context being called. But, ooh! I guess nothing's perfect. Look a the printout from the "catch". It says that it caught the exception thrown by "finally", so what happened to the original exception? Well, it's lost, just like if your "catch" logic were to throw an unintended exception. The only difference is that if your catch logic intends to throw a new exception, it would have the original exception to work with, the "finally" does not. Suffice it to say, "finally" should never "intend" to throw an exception, nor should it be used to contain the bulk of your logic (where exceptions are more likely to lurk).

Our next example shows "finally" being called after "try" throws an exception and "catch" rethrows the exception. Then "finally" also throws an exception.


try
{
button7helper();
}
catch (Exception ex)
{
Console.WriteLine("Example 7: Outer catch: ErrMsg={0}",ex.Message);
return;
}
...
private void button7helper()
{
try
{
Console.WriteLine("Example 7: Try called");
throw new Exception("Example 7: Exception thrown");
}
catch (Exception ex)
{
Console.WriteLine("Example 7: Catch caught: ErrMsg={0}",ex.Message);
Console.WriteLine("Example 7: Catch rethrows exception");
throw ex;
}
finally
{
Console.WriteLine("Example 7: Finally called");
throw new Exception("Example 7: Exception thrown by finally");
}
}


As expected, the try runs, then the catch runs, then the finally runs, then the outer context catch runs. The outer context can only recover the exception info from the "finally".

We haven't touched on threads here. But if its a child thread you are worried about, don't. The try/finally construct is implemented in the compiler and is thread safe. If your thread catches any exceptions, the behavior should be the same as in these examples. But, if your thread lets the exception go un-caught, then the parent thread gets the exception and will have a hard time dealing with it. But, the child thread will at least have cleaned up in the "finally" before the parent dies an ungraceful death.

Oh, and what about the C++ "try/except" construct? This is a C# blog, but I will answer that one, too. The "except" construct is only implemented in C++ and VB (under the name "catch"). It is called a User-Filtered Exception and allows higher level contexts to insert specialized handling into lower level exceptions. In so doing, they insert extra programming from outside of the try/finally context that your code is probably not aware of. It kind of violates the whole purpose of "finally". But that's a C++ and VB problem that C# doesn't have (yet). And the issue and solution are well documented.

Anyhow, here's the example listing. Just create a new Window Forms application, drop 7 buttons on the form, then open the form's code and replace it all with the following...


using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;

namespace TryCatchFinallyExample
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}

private void button1_Click(object sender, EventArgs e)
{
// A simple try finally, the try will run, then the finally, then the remaining code
try
{
Console.WriteLine("Example 1: Try called");
}
finally
{
Console.WriteLine("Example 1: Finally called");
}
Console.WriteLine("Example 1: More code called");
}

private void button2_Click(object sender, EventArgs e)
{
// Now the try returns, never reaching the extra code, but finally still runs
try
{
Console.WriteLine("Example 2: Try called");
return;
}
finally
{
Console.WriteLine("Example 2: Finally called");
}
Console.WriteLine("Example 2: More code called");
}

private void button3_Click(object sender, EventArgs e)
{
// Now the try throws an exception, the exception is caught, finally
// runs, additional code runs
try
{
Console.WriteLine("Example 3: Try called");
throw new Exception("Example 3: Exception thrown");
}
catch (Exception ex)
{
Console.WriteLine("Example 3: Exception Caught: ErrMsg={0}", ex.Message);
}
finally
{
Console.WriteLine("Example 3: Finally called");
}
Console.WriteLine("Example 3: More code called");

}

private void button4_Click(object sender, EventArgs e)
{
// Now the try throws an exception, the exception is caught and return called,
// but finally still runs, however additional code does not run
try
{
Console.WriteLine("Example 4: Try called");
throw new Exception("Example 4: Exception thrown");
}
catch (Exception ex)
{
Console.WriteLine("Example 4: Exception Caught: ErrMsg={0}", ex.Message);
return;
}
finally
{
Console.WriteLine("Example 4: Finally called");
}
Console.WriteLine("Example 4: More code called");
}

private void button5_Click(object sender, EventArgs e)
{
try
{
button5helper();
}
catch (Exception ex)
{
Console.WriteLine("Example 5: Outer catch: ErrMsg={0}",ex.Message);
return;
}
}
private void button5helper()
{
// Now the try throws an exception, but the exception is not caught, finally
// runs, additional code does not
try
{
Console.WriteLine("Example 5: Try called");
throw new Exception("Example 5: Exception thrown");
}
finally
{
Console.WriteLine("Example 5: Finally called");
}
Console.WriteLine("Example 5: More code called");
}

private void button6_Click(object sender, EventArgs e)
{
try
{
button6helper();
}
catch (Exception ex)
{
Console.WriteLine("Example 6: Outer catch: ErrMsg={0}", ex.Message);
return;
}
}
private void button6helper()
{
// Now the try throws an exception, but the exception is not caught, finally
// runs amd also throws and exceptoin, additional code does not run
try
{
Console.WriteLine("Example 6: Try called");
throw new Exception("Example 6: Exception thrown");
}
finally
{
Console.WriteLine("Example 6: Finally called");
throw new Exception("Example 6: Exception thrown by finally");
}
Console.WriteLine("Example 6: More code called");

}

private void button7_Click(object sender, EventArgs e)
{
try
{
button7helper();
}
catch (Exception ex)
{
Console.WriteLine("Example 7: Outer catch: ErrMsg={0}",ex.Message);
return;
}
}
private void button7helper()
{
// Now the try throws an exception, the exception is caught but catch
// throws an exception, finally runs and also throws and exception,
// additional code does not run
try
{
Console.WriteLine("Example 7: Try called");
throw new Exception("Example 7: Exception thrown");
}
catch (Exception ex)
{
Console.WriteLine("Example 7: Catch caught: ErrMsg={0}",ex.Message);
Console.WriteLine("Example 7: Catch rethrows exception");
throw ex;
}
finally
{
Console.WriteLine("Example 7: Finally called");
throw new Exception("Example 7: Exception thrown by finally");
}

}
}
}

Thursday, July 17, 2008

Thread Synchronized Queing

Today's topic gets back to some multi-threading. I find it very useful to have threads pull work from a queue and have one master assignment manager feed the queue. However the standard System.Collections.Generic.Queue is a little inadequate for me because it is not fully thread-safe. I found it a little disheartening that the Queue class members are not overridable either. So we write our own, here are my requirements...


  • The queue must synchronize multiple writer threads writing elements to the queue.

  • The queue must synchronize multiple reader threads reading from the queue.

  • If there is nothing in the queue, a reader thread must wait for something or timeout.

  • Those waiting on the queue can be “interrupted”.

  • Of course, no two readers will ever get the same element off the queue.



  • So let's start coding...

    I start with a new class called SyncQueue in my own name space and I will make it generic.


    namespace QueueExample
    {
    public class SyncQueue
    {
    public void Enqueue(T element)
    {
    // we'll get to it
    }
    public T Dequeue()
    {
    return Dequeue(-1); // wait forever
    }
    public T Dequeue(int timeout_milliseconds)
    {
    // we'll get to this later
    return default(T); // effectively null
    }
    public void Interrupt()
    {
    }
    public T Peek()
    {
    return default(T);
    }
    public int Count { get { return 0; } }
    }
    }


    So there's the basic skeleton of the queue. You can add elements with Enqueue(), remove elements with Dequeue() and even look at the next element with Peek(). There is an operation for interrupting watiting threads called Interrupt(), and for good measures, I will be able to check how many elements are on the queue with a Count property.

    But, how will we synchronize the threads you ask. We will make use of a couple of synchronization tools at our disposal. After adding the name space System.Threading, we have two WaitHandles available to us, AutoResetEvent and ManualResetEvent. So I add them to my class in a private array of WaitHandles called "handles";



    private WaitHandle[] handles = {
    new AutoResetEvent(false),
    new ManualResetEvent(false),
    };


    The other item we will need is a lock so that we can protect the queue from multiple threads tampering with its critical members at the same time. Oh! and lest we forget, we need to add a queue. So I add a System.Collections.Generic.Queue to my class and call it _q.


    private Queue _q = new Queue();



    And now when I need to access a member of _q, I will wrap the critical section with a lock(_q) { } block. So, let's start with something real easy. Let's start with Count.


    public int Count { get { lock(_q){ return _q.Count; } } }



    Count get the _q.Count value after first gaining exclusive access to the queue. The fact is, it really shouldn't matter because as soon as the lock(_q) is released, the count could be changed. This value can be very dynamic so its instantaneous value is of limited use. The lock could be omitted entirely, since interlocking the value won't affect our results.

    Next, lets look at Peek(). It returns the value of the next item on the queue. But, unless there is some form of thread synchronization external to the queue class, peeking at the queue is just for fun because by the time you actually try to dequeue something, the one you peeked will probably be gone. But, here it is anyway, in case you have some additional synchronization you'd like to add.


    public T Peek()
    {
    lock(_q)
    {
    if(_q.Count > 0)
    return _q.Peek();
    }
    return default(T);
    }


    I haven't mentioned it yet, but notice that there is no unlock() that one must call. The lock() is convenient because it effectively implements a try/finally block with a “Monitor” that looks like this...


    Monitor.Enter(_q)
    try {
    //do your stuff
    }
    finally {
    Monitor.Exit(_q);
    }


    Thus it doesn't matter if you return() or throw() an exception, the “finally” will always be called to release the lock. Monitors have some other nice features, but we'll leave that for another day.

    We still haven't used that “handles” array I create nor have we implemented the Enqueue, Dequeue or Interrupt, so let's move on. Next, let's look at Enqueue.


    public void Enqueue(T element)
    {
    lock(_q)
    {
    _q.Enqueue(element);
    ((AutoResetEvent)handles[0]).Set();
    }
    }


    Enqueue locks the queue, adds the element and then signals any thread waiting for a queue element with the AutoResetEvent we setup above. The AutoResetEvent was created in the Reset state so that any thread attempting to remove an item before anything is added will have to wait. By calling the “Set()” method, the Enqueue method is effectively releasing one waiting thread to attempt to lock and read the queue and immediately resets so that only one waiting thread is released.

    Now for Dequeue. It needs to lock the queue, check if there are any elements, and if not, unlock the queue and wait for something to come available. Note that Dequeue cannot lock the queue and then while locked wait for something to come available. With the queue locked, nothing would ever come available.


    public T Dequeue(int timeout_milliseconds)
    {
    T element;
    try
    {
    while(true)
    {
    if(WaitHandle.WaitAny(handles, timeout_milliseconds, true) == 0)
    {
    lock(_q)
    {
    if(_q.Count > 0)
    {
    element = _q.Dequeue();
    if(_q.Count > 0)
    ((AutoResetEvent)handles[0]).Set();
    return element;
    }
    }
    }
    else
    {
    return default(T);
    }
    }
    }
    catch (Exception e)
    {
    return default(T);
    }
    }


    That's a lot more complex than the rest of the methods, so let's look at it in detail starting from the outer most block and working in. We start with a “try” block because we are going to be doing things that are known to cause exceptions from time to time. Specifically, we will be calling WaitAny() which throws an exception if a thread aborts while holding a synchronization handle we need. WaitAny can throw an exception for several other reasons, too. So, we need to deal with the possibility (just returning a null is not the best response, you should expand this section).

    Next step down is a loop where threads compete for the queue and its elements. The first step is to wait. When ever something is added to the queue, the AutoResetEvent is Set, so we wait for something to be added. If something has already been added, the event will be set when the thread gets to this point. The WaitAny call will return 0 if the AutoResetEvent is set or it will return 1 if the ManualResetEvent is Set. It returns a value equal to WaitHandle.WaitTimeout if the operation times out. Therefore we are looking for 0 if we are to possibly take an element from the queue. The other values will result in the else condition being met and the default return value of null being returned. This will be the basics to the operation of the Interrupt() method.

    Now if a 0 is returned, there may be something on the queue and the thread is released to try and lock the queue. It is possible that other items were enqueued as well resulting in multiple threads trying to get to the queue, so the lock(_q) is again necessary. Upon obtaining the lock, there is a possibility that there are no elements on the queue, so we first check the count and only dequeue if the count is greater than 0. If nothing is there, we go back to the WaitAny and wait again. After taking an element off the queue, we again check the count and if there are still elements, we Set the AutoResetEvent so the next waiting thread can proceed.

    Last, we have the Interrupt() method. The Interrupt() method is now very easy.


    public void Interrupt()
    {
    ((ManualResetEvent)handles[1]).Set();
    }


    The Interrupt() method uses the ManualResetEvent to signal waiting threads. This type of event stays signalled until manually reset. So all threads waiting and andy threads coming to the WaitAny will immediately release and fall into the else condition. Note that Interrupt and Timeout are indistinguishable based upon return value. Your thread architecture will need to deal with the higher issues of interrupting threads.

    I moved through the dequeue code pretty quickly, and you may be wondering why the dequeue fires the AutoResetEvent event at all. Maybe you think if it didn't fire the event then there wouldn't be any competition for the queue and then there wouldn't be the need to check if there were anything on the queue. You may ask isn't it guaranteed that there is something on the queue if you get the event signal? No, its not.

    First, consider the very simple and likely case that several items will be added to the queue before any thread gets an element from the queue. The AutoResetEvent will be Set() multiple times, but only allow one thread through, It does not keep track of how many elements there are, it is only a signal to say that there is at least one element on the queue. So, if there are multiple elements on the queue, even though the AutoResetEvent has been Set() several times, the first thread that dequeues results in the AutoResetEvent being Reset(), therefore, it must signal again, or the queue will stop working.

    Now, consider the condition where an item is added to the queue while 3 threads already wait for an element. One thread releases on the event being signaled, but before it can obtain the lock, yet another thread writes another element to the queue and sets the event. This releases the second thread. The two free threads now compete for the lock and only one gets through while the other waits at the lock. Now when the winner takes an element off the queue, it finds another element still there, so it signals the event again according to our first scenario. This releases the third thread to compete with the second thread over the single remaining item on the thread. Therefore, they must check the quantity before they try to take an element. One of the threads will lose and have to go back and wait.

    This queue is rather simple. It does not limit capacity, or seriously block any writers from adding to the queue. The error checking in weak, but all in all, you should be able to gleen a better understanding of the complexities of queue synchonization in a multi-threaded application. And you get some working code to start with to boot! Here is the full listing ...


    using System;
    using System.Collections;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    using System.Threading;

    namespace QueueExample
    {
    public class SyncQueue
    {
    private WaitHandle[] handles = {
    new AutoResetEvent(false),
    new ManualResetEvent(false),
    };
    private Queue _q = new Queue();

    public int Count { get { lock (_q) { return _q.Count; } } }
    public T Peek()
    {
    lock (_q)
    {
    if (_q.Count > 0)
    return _q.Peek();
    }
    return default(T);
    }

    public void Enqueue(T element)
    {
    lock (_q)
    {
    _q.Enqueue(element);
    ((AutoResetEvent)handles[0]).Set();
    }
    }

    public T Dequeue(int timeout_milliseconds)
    {
    T element;
    try
    {
    while (true)
    {
    if (WaitHandle.WaitAny(handles, timeout_milliseconds, true) == 0)
    {
    lock (_q)
    {
    if (_q.Count > 0)
    {
    element = _q.Dequeue();
    if (_q.Count > 0)
    ((AutoResetEvent)handles[0]).Set();
    return element;
    }
    }
    }
    else
    {
    return default(T);
    }
    }
    }
    catch (Exception e)
    {
    return default(T);
    }
    }

    public T Dequeue()
    {
    return Dequeue(-1);
    }

    public void Interrupt()
    {
    ((ManualResetEvent)handles[1]).Set();
    }
    public void Uninterrupt()
    {
    // for completeness, lets the queue be used again
    ((ManualResetEvent)handles[1]).Reset();
    }
    }
    }


    For more on queing, take a look at my article A Simple Task Queue.

    Sunday, July 13, 2008

    Locking Guidelines

    In my last post (Lock Up Unlocked), I gave you some basic guidelines for locking resources in multi-threaded programs.

    Basic guidelines for locking.

  • Avoid simultaneously locking multiple shared resources in a single thread.

  • If multiple shared resources must be locked simultaneously, endeavor that all threads lock the resources in the same order and unlock them in reverse order.

  • If they can't be unlocked in reverse order, then endeavor to unlock in the same order and do not lock an earlier resource till all later locks are released.

  • If you can't follow these guidelines, then timeout.



  • In today's post, I am going to elaborate a little on each principle. But first, let's discuss "resources". Many of you can quickly name the commonly understood resources. Things like memory, cpu, network jump right out, but that's because we are so used to looking at Task Manager and Resource Monitor. Besides, they are a little esoteric, programmers are concerned about specific area of memory when it comes to synchronization and locking, not "memory" as that big fuzzy thing that we don't even manage ourselve anymore. So, "share memory" locations are one resource, a communications port is another.

    But look at it another way. ANYTHING that would cause the thread to wait suggests a resource. Yes, we insert our own locks to make the thread wait for our resources, but there are many operations the thread will perform that could cause the thread to wait. Even a "sleep" operation is a wait for the resource of elapsed time. Or maybe you put your thread to sleep to yield the processor resource for another thread. Reading a file, allocating memory, reading a communications port, all could result in a wait. Writing a file, sending an event, throwing an exception, could result in a wait as well. With C# it has become easy to hook just about everything and with our hooks, we could feasibly insert waits, or blocking calls to obtain resources our event handler needs. In the last post, I pointed out that one of the locked resources in the program was the GUI's event loop. Since every wait is a kind of "lock", it behooves us to know when we are likely to block if we are to accurately apply the "basic guidelines for locking. Now, let's take a look at them.

    Avoid simultaneously locking multiple shared resources in a single thread.
    This one is almost too easy. If a thread only locks one resource, it is never holding a lock some other thread needs while waiting for that lock. Therefore, in is never in the way of another thread. It may compete with other threads, but that's really the name of the game in thread synchronization.

    If multiple shared resources must be locked simultaneously, endeavor that all threads lock the resources in the same order and unlock them in reverse order.
    It can be easily shown that if all threads honor an agreed upon sequence of resource locking, then deadlock is prevented, since no thread can lock resource RB without first locking RA, then the holder of the lock on RA can be sure he's not in the way of some thread holding a lock on RB. This principle becomes particularly important when working with semaphores that allow several threads to access the resource (RA). This is typically done with read-only access to a resource. If write access is needed (RB), read access must first be obtained, preserving the order of locks.

    If they (the resources) can't be unlocked in reverse order, then endeavor to unlock in the same order and do not lock an earlier resource till all later locks are released.
    This approach really only works for applications that need to move through a predefined set of states. If the thread never moves backwards in the states, it can unlock previous locks. Think about a person moving from station to station in a sequence of operations. If the thread must backup, it can't release earlier locks, it must unlock in reverse order.

    If you can't follow these guidelines, then timeout.
    One of the important skills of multi-threaded programming is writing your wait call in such a way as to handle a timeout gracefully. It's even more important when you know you've violated one of the guidelines and therefore will likely deadlock without a timeout. Examine Microsoft's Reader-Writer class and example to see how an attempt is made to gain a write lock, but a timeout is coded for just in case the write lock can't be gotten.

    We started this series with a simple deadlock on what looked like a critical sections. "Locking" and its twin brother "waiting" can do a whole lot more than critical sections. But, how then was the deadlock solved? Yes, we could have just removed the locks, but that would have been no fun. The deadlock was prevented when the child thread chose not to lock RB while holding the lock on RA. BeginInvoke() is analogous to the Post Event call of the old days. It does not wait for the GUI thread to service the event. The child thread then releases the lock (RA), then tries to lock only RB by calling EndInvoke() to wait for completion of the event. Since the first guideline is not violated, there is no deadlock.

    No code today.

    Saturday, July 12, 2008

    Lock Up Unlocked

    In my last post (Locked Up!), I gave an example of a simple deadlocking application. If you run it and click the button a few times rapidly, the application stops responding. Then, if you pause the program and examine the stack trace of each thread, you find the GUI thread waiting for the lock, and the child thread waiting for Invoke() to return. The child thread is holding the lock, so the application is deadlocked. This is also known as a "deadly embrace".

    So, I affectionately referred to the two resources as Resource RA and Resource RB and asked you to identify them in the program. One of the resources is pretty obvious. That would be the lblResult.Text. It doesn't matter whether you call this RA or RB, so I'll call it Resource RA. Regardless, for some odd reason, the programmer has decided that the lblResult.Text property needs to be protected from simultaneous read and write. This protection is not really necessary, but pretend for a moment a valid sort of resource did indeed need this kind of protection. Thus, we have the lock on Resource RA as lock(lockObject) { ... }. Were you going to say lockObject is RA? That's okay, that can work, too. But, as I'll try to show, its preferable to think of the lockObject as part of the mechanism for locking a resource, and consider the code or resource being protected as the true resource.

    Determining Resource RB is more difficult, since there are no other locks in the code to make it stand out. There is only the lock around the reading and the lock around the writing of the text field. The key to figuring out what Resource RB is is to look for where the program is waiting or blocking.

    The child thread is stuck waiting for Invoke() to complete. Invoke() is not the resource and technically, its not the lock either. But, none the less, that is the point at which the child appears blocked. The child is waiting for a resource to come free. The Resource RB is then the GUI thread's processing of the child thread's request. But, the GUI thread did not lock a resources, or did it?

    The GUI thread is regularly returning to the event loop to process the next event on the queue or wait until there are events (it's more complex than this, but we'll keep it simple for the sake of the discussion). A queue such as this is a locking mechanism. While the GUI is busy, servicing an event, anyone waiting for a particular event to complete is blocked until the GUI pulls the event and processes the event being waited for. You can think of the GUI thread as having locked RB by pulling an event and working on it. In this simple case, the GUI thread actually pulled the child thread's event (Invoke request) and tried to work on it. But, the child had locked the resource the GUI needed to continue. Consider the more complex case where the GUI pulls some other event for some other thread waiting for service and the problem could become daunting.

    Now, the whole purpose in thinking about this problem as dealing with just two resources RA and RB is to present a simple analytic approach. You may have hit a deadlock or two in your novice foray into multi-threaded applications. Having hit a deadlock or related race condition, you may not have had the analytic tools to diagnose the problem. So start with the first tool called "understanding". The basic or classic deadlock or deadly-embrace occurs when one thread holds a resource (RA) and needs another resource (RB) while another thread holds the second resource (RB) and needs the first resource (RA). If you can understand this, and if you can boil the problem down to this simple formula, you can begin solving the problem with basic locking principles. As you advance in multi-threaded programming, you'll learn more and more of the locking patterns and how to apply them. But, in the mean time, if you boil everything down to simply "locking a resource", some basic guidelines will help you correct your logic or design.

    Basic guidelines for locking.

  • Avoid simultaneously locking multiple shared resources in a single thread.

  • If multiple shared resources must be locked simultaneously, endeavor that all threads lock the resources in the same order and unlock them in reverse order.

  • If they can't be unlocked in reverse order, then endeavor to unlock in the same order and do not lock an earlier resource till all later locks are released.

  • If you can't follow these guidelines, then timeout.



  • There's more to be said about these basic locking principles, and I'd like to touch on the touchy subject of knowing one's resources. But, I'll save that for the next post and instead leave you with some code that breaks the deadlock and hopefully provokes some more thought about the often overlooked event loop "resource". Maybe you can tell why this breaks the deadlock based upon the RA/RB discussion.

    Here's the code.


    using System;
    using System.Collections.Generic;
    using System.ComponentModel;
    using System.Data;
    using System.Drawing;
    using System.Linq;
    using System.Text;
    using System.Windows.Forms;
    using System.Threading;
    // Now let's see how we can avoid locking ourselves up
    //
    namespace LockupExample
    {
    public partial class Form1 : Form
    {
    public Form1()
    {
    InitializeComponent();
    _t = new Thread(new ParameterizedThreadStart(WorkerThread));
    }

    private Thread _t;
    private object lockObject = new object();
    private bool StopThread = false; // to let the thread quit gracefully
    private void WorkerThread(object sender)
    {
    Thread.Sleep(1000);
    while (!StopThread)
    {
    string result = "This is a Test";
    IAsyncResult aResult;

    lock (lockObject)
    {
    Thread.Sleep(25);
    aResult = lblResult.BeginInvoke(new MethodInvoker(delegate { lblResult.Text = result; }));
    }
    lblResult.EndInvoke(aResult);
    Thread.Sleep(500);
    }
    }

    private void Form1_Load(object sender, EventArgs e)
    {
    _t.Start();
    }

    private void button1_Click(object sender, EventArgs e)
    {
    lock (lockObject)
    {
    lblResult.Text = "Override the Test";
    }
    }

    private void Form1_FormClosing(object sender, FormClosingEventArgs e)
    {
    StopThread = true;
    }
    }
    }

    Friday, July 11, 2008

    Locked Up!

    Today's example is NOT useful code. Instead, it's a simple example of a bad practice in multi-threaded code floating around out there in programming land. This example is C# specific, but the concepts apply to other languages as well.

    Since this is a bad example, I'll start off by telling you why its bad. The application locks then blocks on another resource. It's as simple as that. Many of you understand that if Thread A locks Resource RA and tries to lock Resource RB, while Thread B locks Resource RB and then tries to lock Resource RA, you have a classic "deadlock".

    The following code "deadlocks". Run it and click the button a few times and it will deadlock. So, the real question is not "why is it deadlocking?" I just told you up front, it's a classic deadlock. The real question is "what are the equivalents in this program of Resources RA and RB?"

    To answer this question, you can run the program in your favorite debugger and click the button a few times until it deadlocks (lengthen the sleep inside the child thread lock if need be). "Pause" your debugger to see where the threads have stopped.

    So set up your project (I use VS 2008). Create a new Windows Form project, call it LockupExample. On Form1, drop a Label control and rename it lblResult. Also, on the form, drop a Button control (leave it button1). Now view the form's code, select all and replace with the following...


    using System;
    using System.Collections.Generic;
    using System.ComponentModel;
    using System.Data;
    using System.Drawing;
    using System.Linq;
    using System.Text;
    using System.Windows.Forms;
    using System.Threading;
    // Let's see how we can lock ourselves up
    //
    namespace LockupExample
    {
    public partial class Form1 : Form
    {
    public Form1()
    {
    InitializeComponent();
    _t = new Thread(new ParameterizedThreadStart(WorkerThread));
    }

    private Thread _t;
    private object lockObject = new object();
    private void WorkerThread(object sender)
    {
    Thread.Sleep(1000);
    while (true)
    {
    string result = "This is a Test";
    lock (lockObject)
    {
    Thread.Sleep(25);
    lblResult.Invoke(new MethodInvoker(delegate { lblResult.Text = result; }));
    }
    Thread.Sleep(500);
    }
    }

    private void Form1_Load(object sender, EventArgs e)
    {
    _t.Start();
    }

    private void button1_Click(object sender, EventArgs e)
    {
    lock (lockObject)
    {
    lblResult.Text = "Override the Test";
    }
    }
    }
    }


    I'll discuss the answer in my next post.