Issue 14

(Rather than just reiterating the new features in ADO.NET that we announced for //Build/, I figured that I’d do a series of posts covering various features in depth – although this first "feature" shipped a bit earlier than the 4.5 Developer Preview)

What’s in a fix

If you remember last month’s Patch Tuesday, the first Reliability Update for .Net 4.0 was release, including a bug fix for System.Data.dll. However, those of you who read the support article would have been greeted by this cryptic message:

Issue 14
Consider the following scenario

  • You use the .NET Framework Data Provider for SQL Server (SqlClient) to connect to an instance of Microsoft SQL Azure or of Microsoft SQL Server.
  • An established connection is removed from the connection pool.
  • The first request is sent to the server.

In this scenario, an instance of SqlException is encountered, and you receive the following error message:
A transport-level error has occurred when sending the request to the server.

So given that description, can you tell what the original bug was, or what we fixed?
No? Neither can I – and I wrote the fix…

Historical Perspective

To explain "Issue 14", we’ve first got to look back at the history of ADO.NET, back to the 2.0 (or, possibly, 1.0) days. In the original design of the Connection Pool it was decided that, if there was a catastrophic failure of a connection, then the entire pool should be cleared. This was a reasonable assumption, since being unable to communicate with the server typically means that either the server is down (or restarted), the client network connection has died or failover has occurred – in any of these circumstances, it is unlikely that any other connection in the pool had survived.

Fast forward to today, and some of the original assumptions of the connection pool are no longer valid. Due to the increased popularity of cloud computing and connected devices, connections to SQL Servers are might not be going over ultra-fast and ultra-reliable links inside a data center. Instead, they may be going over the internet, which means that they are unreliable and could drop at any time. This, combined with SQL Azure’s policy of dropping connections that have been idle for over 30 minutes, meant that we could have one dead connection in the pool, but the rest would be ok.

Check Connections

So now we’re connecting over unreliable connections and still clearing the pool when it’s possible that only one of the connection had died. On top of that, we don’t know the connection is dead until someone tries to execute a command on the connection (and then gets an error, despite the fact that they had just opened the connection). So what to do?

Firstly, we are now checking the state of the connection when we remove it from the connection pool and, if its dead, giving you a new connection. This greatly decreases the likelihood that you will be trying to execute on a bad connection (although its still possible, as we are relying on Windows to know about the state of the underlying TCP connection, and since there is a race condition between us checking and you executing on the connection).

Secondly, we no longer clear the pool when there is a fatal error on a connection – so we’re no longer dropping (hopefully) good connections just because one connection is bad. Conversely, since we are checking connections before using them, we are still responsive to events like failover or network disconnects.

Best Practices

If you read the last section carefully, you would have noticed one of the caveats of this feature: "there is a race condition between us checking and you executing on the connection", which leads to my first recommendation:

Follow the Open-Execute-Close pattern

Every time you need to interact with SQL Server, you should open a fresh connection, execute your command (and deal with the data reader if you open one) and then close the connection (since SqlConnection, SqlCommand and SqlDataReader are all implement IDisposable, the best way to do this is with a ‘using’ statement). If you need to expose the data reader to a higher level API, and don’t want to cache the data in the data reader, then you should wrap the connection, command and reader inside a new class that implements IDisposable and return that instead.

My second recommendation relates back to my previous post on connection strings:

Use our connection pool

Despite the connection pooling code being rather old, it is extremely fast, reliable and it works. Opening connections from the pool and returning them afterwards is incredibly quick, especially when compared to opening a fresh connection or executing a command on a connection. Additionally we have the ability to introduce features like this which custom pooling code can’t.

Finally, this improvement is no replacement for proper retry code.

Improvements in 4.5

In the .Net 4.5 Developer Preview, we’ve made this feature more scalable, especially for high-throughput application servers where connections do not sit idle in the pool for very long.

As a final note, if you haven’t already upgraded to .Net 4.5, then you should make sure that you’ve installed the 4.0 Reliability Update.

Tagged , , , , , ,

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: