Thursday 16 April, 2009 (BizTalk | Fixes)
These last few days I have been having enormous problems with my BizTalk development environment running in a windows Server 2003 Virtual PC environment. Whenever the BizTalkServerApplication host was started CPU usage would stay at 100% consistently. Stopping all my applications and even deleting them changed nothing. No running or suspended instances were visible. For some reason the sqlservr.exe process was consuming all available CPU resources. I found a couple of links that helped me solve the problem:
http://www.tech-archive.net/Archive/BizTalk/microsoft.public.biztalk.general/2006-11/msg00243.html
http://www.digitaldeposit.net/saravana/post/2008/06/04/Read-this-before-using-bts_CleanupMsgBox-stored-procedure.aspx
http://technet.microsoft.com/en-us/library/bb727781.aspx
Basically running the following SQL:
After installing the script according to the instructions provided by Microsoft solved my issues! I suspect that this could have to do with a problem with zombie messages that I have had.
There is also a hotfix available in case you have the wrong version installed. http://support.microsoft.com/kb/924715
Wednesday 12 November, 2008 (.Net | BizTalk | Ramblings | Team System | Tech Ed 2008 | Visual Studio)
This is my third day at Tech Ed 2008 in Barcelona, the breadth of the technologies and languages that we as .Net developers now need to at least understand and preferably master to some degree is enormous. I believe that developers will need to specialize more explicitly on certain technology areas to remain productive in the future. It has been impossible to master all of Microsoft's products for many years now but up until a year or so ago it was at least conceivable for a developer to master all of the developer technologies that Microsoft had on offer. With the introduction of cloud based services and all the news in .Net, both libraries and new languages I cannot see how any one person will be able to master it all... I want to sum of some of my thoughts on the sessions that I have attended here at Tech Ed on Visual Studio 2010.
Fixing Continuous Integration
CI is a great principle and has moved into the mainstream along with many other agile principles. While Continuous Integration as such has never been broken there has always been a problem with developer discipline. If you check in something that is broken then it will be discovered quickly with CI but if you don't fix the broken code other developers will be less productive if they cannot get the latest code or have their own checkins fail in CI due to previously broken builds. With Visual Studio Team System 2010 Microsoft is introducing a new concept for CI, the gated check. It is not a big change to the CI flow but a very clever use of a shelving, a feature in TFS which I find generally is underutilized. Instead of doing a standard checking in and then running a build on that code, gated checkins shelve the developers changes and then performs a build on the shelveset. If this build is successful then the shelveset is promoted to a proper check in, if the build fails then the check in is not committed to the mainline. The visibility that CI offers is maintained but using gated checkins other developers will never get failing code from other developers which was discoverable! Obviously undiscovered errors can still sneak into the code due to bad or missing tests or incorrect configuration of the build environment but it I believe that this will lower or eliminate the fear of checking in that some developers have. It will also allow you to check in before you go home without the need to wait and see that the CI build completes successfully. If I check in my code 5 minutes before I go home I should ideally wait to see that the build is successful, if it does not then I need to fix that immediately since other developers will be getting that code tomorrow morning when they come in. Using a gated check in I can go home, let the CI build complete on my shelved changes and if the build is successful then everyone can get my changes in the morning, if it fails I can safely wait until the morning before fixing the code and no one else is impacted by my errors.
Reproduction of Errors and Architecture Compilation Failures
Visual Studio 2010 looks like it contains some pretty good testing tools like the ability to record a testers GUI while performing manual tests and also including debugging and tracing info from remote machines. Pretty powerful stuff! It seems Microsoft have their mind set on eliminating the problem of irreproducible errors.
Another interesting feature is the architecture layer diagram which allows you to specify how the different layers in your application may and may not call each other. This is useful for validating that your code follows the intended architecture at compile time instead of detecting it when problems arise. All of these features (and there are more) are great but unfortunately Microsoft still has silly distinctions between different editions of Visual Studio Team System, so many of the cool demo features will probably be marginalized in most organisations due to them not actually having the appropriate edition of Visual Studio.
Monday 03 November, 2008 (.Net | BizTalk | Fixes | Java)
Working with Websphere MQ and the MQSC adapter is quite new for me and I have been struggling with a couple of problems which have been very difficult to find solutions to. One of our main issues has been getting the correct character encoding of outgoing messages to MQ. BizTalk would output the message with the correct encoding (using a file send port to verify) and it looked like the messages on the WebSphere MQ queue were correctly encoded when we checked them using rhutilc. In our case we encoded our unicode messages as UTF-8 but when the Java system which is the destination read the messages any non-standard US characters came out as rubbish. They were correctly encoded as two bytes in the message but the destination system was reading them using codepage 850. Also all unicode messages that the Java system sent to us encoded with UTF-8 over MQ worked correctly in BizTalk.
What I have found out through much trial and error and the sparse clues I have been able to find on the Internet is that the WebSphere client that BizTalk uses under the covers will set the character encoding of any messages that you write to the systems default codepage, which for windows is 850 according to IBM. This setting does not take into account the actual encoding that BizTalk 2006 has done to the message. If you force your message to be encoded as 850 in BizTalk then it will most likely work as long as you don't use characters that are outside the codepage. There is a context property in MQSeries.dll called MQMD_CodedCharSetId which will allow you to set the codepage value (CCSID in the MQ world) for your outgoing message.
I have found three ways to remedy the encoding problem:
- Set the context property in an orchestration to whatever value you require. The drawback with this is that if you need to change your encoding you have to recompile and redeploy everything. Also changing your context property does not actually change the encoding just how MQ should interpret the message.
- Set an MQ environment variable named MQCCSID to the codepage that you send your messages as. This will override the system default value and force messages to be tagged with your required codepage. Obviously you can only have one system wide default so if you need to send messages out with different encodings this isn't going to help. Again the actual encoding of your messages is not going to be changed.
- Create an encoding pipeline component that allows you to set the MQ codepage for an outgoing message. This is the option that we have settled on for now. The encoding of your message is still not actually changed, unless you implement it, but now you can have different encodings for different destinations and you can change the encoding value using MMC and binding files.
One important point to remember is that UTF-8 is codepage 65001 in Windows but 1208 in IBM, so wherever you set the codepage you need to have this in mind - as far as I know all other codepages have the same ids.
Using rfhutilc you can verify the codepage of a message in an MQ queue by browsing to the message and then checking what codepage has been set under the MQMD tab.
I have not verified this but the solutions that I have outlined above should work for EBCDIC and any other odd encodings that you may require as long as you can actually create a message with that encoding.
Friday 10 October, 2008 (.Net | BizTalk | Fixes | Tips)
The BOM is a two to four byte sequence at the start of an encoded file which tells the recipient if the characters are encoded using big-endian or little-endian byte order. In BizTalk generally the byte order mark is added when a file is UTF-8 or UTF-16 encoded using an XML assembler in a send pipeline. Sometimes applications cannot handle the initial bytes and you don't want them to be added to your message. I recently read two articles about removing the Byte Order Mark (BOM) from outgoing messages in BizTalk 2006 one by Saravana Kumar and the other an official Microsoft KB article. Both of them recommend creating a custom pipeline with an XML assemble stage where the PreserveBom property is set to false. I find it a little strange that they don't point out that you can accomplish this using the default XMLTransmit pipeline also. Just open the pipeline configuration in the Send Port properties and set PreserveBom to false... No custom Pipeline, no coding required.
Friday 04 July, 2008 (BizTalk | Security | Tips)
There is a nice little SFTP adapter for BizTalk on CodePlex http://www.codeplex.com/SftpAdapter/. I have been trying this out for a couple of days and it works quite nicely. I have been using freeSSHd as SFTP server and WinScp as a client to see that the SFTP functionality works. WinScp comes with PuttyGen for generating key pairs for authentication and there are a couple of guides on the web which offer clues as to how this should all be configured but I still had to resort to trial and error to get the SFTP authentication working. So this is my super concise guide to getting it up and running for local testing (presuming you have installed all the software first).
Generate a keypair with PuTTYGen
Under WinScp in the start menu you have "Key Tools/PuTTYGen"
Click "Generate" and move the mouse about. When PuTTYGen has created your keypair enter a comment so that you key is easier to identify, you can use a passphrase for your master key if you like. Save your private key as a .ppk file somewhere safe this is your master file and you can open it in PuTTYGen later if you need to export it again. Export the key (under the Conversions menu) as an OpenSSH key, but remove the passphrase first since the SFTP adapter doesn't currently support passphrases for private keys, save this key somewhere where the SFTP adapter can access it. This key should also be kept safe since it is the equivalent of a password. Select the entire textbox with the public key and copy to the clipboard.
Configure freeSSHd for public key authentication
Open up the freeSSHd settings dialogue and go to Authentication. Enter a directory where the public keys for your users will be stored. Make sure that "Public Key Authentication" is not set to disabled and that "Password Authentication" isn't required.
You also need to set a directory under the SFTP tab for storing the files. Add users in the Users tab selecting "Public key (SSH only)" as the authorization method. Also make sure that the SFTP checkbox is selected.
Go to the directory that has been configured for freeSSHd keys. Create a text file named "username" ( the users login name and no extension) and paste the public key string from PuTTYGen into it making sure it is all on one line.
Test it with WinScp before trying it from BizTalk. Now you should be up and running with your SFTP adapter and an SFTP server.
Page 1 of 1 in the BizTalk category
Disclaimer
The opinions expressed herein are my own personal opinions and do not represent
my employer's view in any way.
Sign In