Reflections on three days at Tech Ed

This is my third day at Tech Ed 2008 in Barcelona, the breadth of the technologies and languages that we as .Net developers now need to at least understand and preferably master to some degree is enormous. I believe that developers will need to specialize more explicitly on certain technology areas to remain productive in the future. It has been impossible to master all of Microsoft's products for many years now but up until a year or so ago it was at least conceivable for a developer to master all of the developer technologies that Microsoft had on offer. With the introduction of cloud based services and all the news in .Net, both libraries and new languages I cannot see how any one person will be able to master it all... I want to sum of some of my thoughts on the sessions that I have attended here at Tech Ed on Visual Studio 2010.

Fixing Continuous Integration

CI is a great principle and has moved into the mainstream along with many other agile principles. While Continuous Integration as such has never been broken there has always been a problem with developer discipline. If you check in something that is broken then it will be discovered quickly with CI but if you don't fix the broken code other developers will be less productive if they cannot get the latest code or have their own checkins fail in CI due to previously broken builds. With Visual Studio Team System 2010 Microsoft is introducing a new concept for CI, the gated check. It is not a big change to the CI flow but a very clever use of a shelving, a feature in TFS which I find generally is underutilized. Instead of doing a standard checking in and then running a build on that code, gated checkins shelve the developers changes and then performs a build on the shelveset. If this build is successful then the shelveset is promoted to a proper check in, if the build fails then the check in is not committed to the mainline. The visibility that CI offers is maintained but using gated checkins other developers will never get failing code from other developers which was discoverable! Obviously undiscovered errors can still sneak into the code due to bad or missing tests or incorrect configuration of the build environment but it I believe that this will lower or eliminate the fear of checking in that some developers have. It will also allow you to check in before you go home without the need to wait and see that the CI build completes successfully. If I check in my code 5 minutes before I go home I should ideally wait to see that the build is successful, if it does not then I need to fix that immediately since other developers will be getting that code tomorrow morning when they come in. Using a gated check in I can go home, let the CI build complete on my shelved changes and if the build is successful then everyone can get my changes in the morning, if it fails I can safely wait until the morning before fixing the code and no one else is impacted by my errors.

Reproduction of Errors and Architecture Compilation Failures

Visual Studio 2010 looks like it contains some pretty good testing tools like the ability to record a testers GUI while performing manual tests and also including debugging and tracing info from remote machines. Pretty powerful stuff! It seems Microsoft have their mind set on eliminating the problem of irreproducible errors.

Another interesting feature is the architecture layer diagram which allows you to specify how the different layers in your application may and may not call each other. This is useful for validating that your code follows the intended architecture at compile time instead of detecting it when problems arise. All of these features (and there are more) are great but unfortunately Microsoft still has silly distinctions between different editions of Visual Studio Team System, so many of the cool demo features will probably be marginalized in most organisations due to them not actually having the appropriate edition of Visual Studio.

Comments [1]

BizTalk 2006 MQSC Adapter Encoding Solution

Monday 03 November, 2008 (.Net | BizTalk | Fixes | Java)

Working with Websphere MQ and the MQSC adapter is quite new for me and I have been struggling with a couple of problems which have been very difficult to find solutions to. One of our main issues has been getting the correct character encoding of outgoing messages to MQ. BizTalk would output the message with the correct encoding (using a file send port to verify) and it looked like the messages on the WebSphere MQ queue were correctly encoded when we checked them using rhutilc. In our case we encoded our unicode messages as UTF-8 but when the Java system which is the destination read the messages any non-standard US characters came out as rubbish. They were correctly encoded as two bytes in the message but the destination system was reading them using codepage 850. Also all unicode messages that the Java system sent to us encoded with UTF-8 over MQ worked correctly in BizTalk.

What I have found out through much trial and error and the sparse clues I have been able to find on the Internet is that the WebSphere client that BizTalk uses under the covers will set the character encoding of any messages that you write to the systems default codepage, which for windows is 850 according to IBM. This setting does not take into account the actual encoding that BizTalk 2006 has done to the message. If you force your message to be encoded as 850 in BizTalk then it will most likely work as long as you don't use characters that are outside the codepage. There is a context property in MQSeries.dll called MQMD_CodedCharSetId which will allow you to set the codepage value (CCSID in the MQ world) for your outgoing message.

I have found three ways to remedy the encoding problem:

Set the context property in an orchestration to whatever value you require. The drawback with this is that if you need to change your encoding you have to recompile and redeploy everything. Also changing your context property does not actually change the encoding just how MQ should interpret the message.
Set an MQ environment variable named MQCCSID to the codepage that you send your messages as. This will override the system default value and force messages to be tagged with your required codepage. Obviously you can only have one system wide default so if you need to send messages out with different encodings this isn't going to help. Again the actual encoding of your messages is not going to be changed.
Create an encoding pipeline component that allows you to set the MQ codepage for an outgoing message. This is the option that we have settled on for now. The encoding of your message is still not actually changed, unless you implement it, but now you can have different encodings for different destinations and you can change the encoding value using MMC and binding files.

One important point to remember is that UTF-8 is codepage 65001 in Windows but 1208 in IBM, so wherever you set the codepage you need to have this in mind - as far as I know all other codepages have the same ids.

Using rfhutilc you can verify the codepage of a message in an MQ queue by browsing to the message and then checking what codepage has been set under the MQMD tab.

I have not verified this but the solutions that I have outlined above should work for EBCDIC and any other odd encodings that you may require as long as you can actually create a message with that encoding.