Thursday 16 April, 2009 (BizTalk | Fixes)
These last few days I have been having enormous problems with my BizTalk development environment running in a windows Server 2003 Virtual PC environment. Whenever the BizTalkServerApplication host was started CPU usage would stay at 100% consistently. Stopping all my applications and even deleting them changed nothing. No running or suspended instances were visible. For some reason the sqlservr.exe process was consuming all available CPU resources. I found a couple of links that helped me solve the problem:
http://www.tech-archive.net/Archive/BizTalk/microsoft.public.biztalk.general/2006-11/msg00243.html
http://www.digitaldeposit.net/saravana/post/2008/06/04/Read-this-before-using-bts_CleanupMsgBox-stored-procedure.aspx
http://technet.microsoft.com/en-us/library/bb727781.aspx
Basically running the following SQL:
After installing the script according to the instructions provided by Microsoft solved my issues! I suspect that this could have to do with a problem with zombie messages that I have had.
There is also a hotfix available in case you have the wrong version installed. http://support.microsoft.com/kb/924715
Monday 03 November, 2008 (.Net | BizTalk | Fixes | Java)
Working with Websphere MQ and the MQSC adapter is quite new for me and I have been struggling with a couple of problems which have been very difficult to find solutions to. One of our main issues has been getting the correct character encoding of outgoing messages to MQ. BizTalk would output the message with the correct encoding (using a file send port to verify) and it looked like the messages on the WebSphere MQ queue were correctly encoded when we checked them using rhutilc. In our case we encoded our unicode messages as UTF-8 but when the Java system which is the destination read the messages any non-standard US characters came out as rubbish. They were correctly encoded as two bytes in the message but the destination system was reading them using codepage 850. Also all unicode messages that the Java system sent to us encoded with UTF-8 over MQ worked correctly in BizTalk.
What I have found out through much trial and error and the sparse clues I have been able to find on the Internet is that the WebSphere client that BizTalk uses under the covers will set the character encoding of any messages that you write to the systems default codepage, which for windows is 850 according to IBM. This setting does not take into account the actual encoding that BizTalk 2006 has done to the message. If you force your message to be encoded as 850 in BizTalk then it will most likely work as long as you don't use characters that are outside the codepage. There is a context property in MQSeries.dll called MQMD_CodedCharSetId which will allow you to set the codepage value (CCSID in the MQ world) for your outgoing message.
I have found three ways to remedy the encoding problem:
- Set the context property in an orchestration to whatever value you require. The drawback with this is that if you need to change your encoding you have to recompile and redeploy everything. Also changing your context property does not actually change the encoding just how MQ should interpret the message.
- Set an MQ environment variable named MQCCSID to the codepage that you send your messages as. This will override the system default value and force messages to be tagged with your required codepage. Obviously you can only have one system wide default so if you need to send messages out with different encodings this isn't going to help. Again the actual encoding of your messages is not going to be changed.
- Create an encoding pipeline component that allows you to set the MQ codepage for an outgoing message. This is the option that we have settled on for now. The encoding of your message is still not actually changed, unless you implement it, but now you can have different encodings for different destinations and you can change the encoding value using MMC and binding files.
One important point to remember is that UTF-8 is codepage 65001 in Windows but 1208 in IBM, so wherever you set the codepage you need to have this in mind - as far as I know all other codepages have the same ids.
Using rfhutilc you can verify the codepage of a message in an MQ queue by browsing to the message and then checking what codepage has been set under the MQMD tab.
I have not verified this but the solutions that I have outlined above should work for EBCDIC and any other odd encodings that you may require as long as you can actually create a message with that encoding.
Thursday 30 October, 2008 (dasBlog | Fixes | Reviews | Team System | Visual Studio | Web)
I have been messing about with some template experiments in dasBlog using Notepad++ and Visual Studio 2008. That is changing the site layout by editing homeTemplate.blogtemplate, dayTemplate.blogtemplate and itemTemplate.blogtemplate for a dasBlog Theme. These files are basically html files with dasBlog specific macros for inserting the blog functionality.
Notepad++
First off I love Notepad++ as a replacement for Notepad. I have had a hard time doing more complex things in Notepad++ due to my addiction to Microsoft's intellisense... Editing the .blogtemplate files is dead simple in Notepad++ since it is very easy to get syntax coloring - you simply choose a language (html) from the language menu and viola everything is color coded. The file that you are editing doesn't need to have the correct extension or even be written in the language you choose. Notepad++ does its best and highlights whatever it can recognize.
There is also an auto completion function that you access by pressing CTRL+space, but this function is really only a text completion function and does not take any notice of the context that you are typing in. It just pops up an alphabetical list of potential text candidates based on the letters that you have already typed. For simple programming tasks this is more than adequate and Notepad++ does have a significantly smaller footprint than Visual Studio, so you can install it on pretty much any computer or server without anyone complaining for quick debugging tasks.
What I really have been missing is proper html intellisense for the .blogtemplate files.
Visual Studio 2008
Obviously Visual Studio does not have support for the .blogtemplate extension. For awhile I hacked my way around it by renaming the files to have a html extension while editing, which sort of works. The main problem is that I keep forgetting to change the extension back before I start testing my changes - so this was causing unnecessary pain... I had a vague memory of configuring something like this before and found these instructions. For the sake of completeness I include them here also:
1. Go to the Tools->Options menu.
2. Pick Text Editor -> File Extension from the tree in the left part of the Options dialog.
3. Type your file extension, .blogtemplate in my case, in the Extension text box.
4. Select the appropriate editor from the Editor dropdown.
5. Click Add and then Ok to close the dialog and re-open your files.
If you cannot see the Text Editor node in the tree view of the Options dialog then you probably need to click on Show all settings in the bottom left hand corner of the dialog...
Using this feature you can set up pretty much any color coding and intellisense support that you like in Visual Studio for file types that Visual Studio lacks support for. All I am missing now is auto completion for dasBlog macros in .blogtemplate files, this seems like a much tougher problem.
Friday 10 October, 2008 (.Net | BizTalk | Fixes | Tips)
The BOM is a two to four byte sequence at the start of an encoded file which tells the recipient if the characters are encoded using big-endian or little-endian byte order. In BizTalk generally the byte order mark is added when a file is UTF-8 or UTF-16 encoded using an XML assembler in a send pipeline. Sometimes applications cannot handle the initial bytes and you don't want them to be added to your message. I recently read two articles about removing the Byte Order Mark (BOM) from outgoing messages in BizTalk 2006 one by Saravana Kumar and the other an official Microsoft KB article. Both of them recommend creating a custom pipeline with an XML assemble stage where the PreserveBom property is set to false. I find it a little strange that they don't point out that you can accomplish this using the default XMLTransmit pipeline also. Just open the pipeline configuration in the Send Port properties and set PreserveBom to false... No custom Pipeline, no coding required.
Wednesday 08 October, 2008 (.Net | Bugs | Fixes | Vista | Web)
While trying out a complex URL rewriting scenario in ASP.Net I ran into a pretty strange problem that resulted in a 404 error:
Description: HTTP 404. The resource you are looking for (or one of its dependencies) could have been removed, had its name changed, or is temporarily unavailable. Please review the following URL and make sure that it is spelled correctly.
Requested Url: /6.aspx/t.aspx
My code is very similar to a solution that I have created previously in ASP.Net 2.0 on Windows XP so I was a little surprised that I was getting an error. The strange thing is that /6.aspx is the original URL and /default.aspx is the rewritten url, in the error message it looks like the original URL has partially overwritten the new URL!
The exception that is caught has a stack trace that ends with: System.Web.UI.Util.CheckVirtualFileExists(VirtualPath virtualPath)
Looking around the Internet for people who have already encountered this problem (as I always do) didn't lead to any immediate solutions. On the IIS forum there is one discussion on Module doesn't work on IIS7 under vista business which lead me to a solution. It seems to me that URL rewriting is broken on Vista prior to SP1. Part of the SP1 upgrade incorporates changes to IIS 7 that were made in Windows Server 2008 as detailed on Microsoft TechNet. So after installing SP1 my solution works as expected.
Thursday 26 June, 2008 (.Net | Fixes | Ramblings | Web)
I was just surfing around on http://forums.microsoft.com and got a yellow screen of death!
I must say I was slightly surprised. In web projects that I have been involved in we have always added two customErrors pages one to handle generic errors which redirects to an errorpage.aspx page and then a static html page errorpage.html which is only used when an error occurs on the dynamic error page... A sample configuration using location follows:
<configuration>
<system.web>
<customErrors mode="on" defaultRedirect="errorpage.aspx"/>
</system.web>
<location path="errorpage.aspx">
<system.web>
<customErrors mode="on" defaultRedirect="errorpage.html"/>
</system.web>
</location>
</configuration>
That way if we have a problem with our application that affects the entire site, for instance menu and navigational rendering, we can still show a nice error page with a message like "The site is temporarily unavailable, please try again in a few minutes."
Thursday 26 June, 2008 (.Net | Bugs | Fixes | Team System)
We have just had a major Active Directory issue that has affected the Team Foundation Server instance that all the developers at the client that I am working at use. The problem occured when some users where erased in the Active Directory and were subsequently recreated with the same user names. These new user accounts cannot access the Team Foundation Server.
A large part of the problem was solved by just removing users from TFS and then putting them back in again. This recreates all the wiring under the surface in TFS so that the users can access the server but all their existing workspace info, pending checkins and so on are lost locally yet exist in Team Foundation Server with a different Owner under the same user name... The only way to tell them apart is by their SID and that the original account may get suffixed with a number in workspace lists: AD\username:61.
My first thought was to remove all the workspaces and pending checkins and let people manually resynch their projects. When I clicked undo pending change for a bunch of changes in Attrice Sidekicks an unhandled exception popped up: "Object reference not set to an instance of an object". I just can't undo these changes, there is a clue to the problem when I try to unlock a changed file, I get an Unlock error:
Failed to unlock $/Project/Path/file (TF14061: The workspace COMPUTERNAME; AD\username does not exist.)
I can't search for workspaces by username either, even doing a simple TFSSecurtiy /i "ad\username" /server:TFSName for one of the affected accounts returns "Error: The identity cannot be resolved." Since the original accounts have been removed from Active Directory I cannot easily get hold of their SIDs for querying with TFSSecurity. Attempting to delete the workspaces from the command line return the following error:
TF14061: The workspace WORKSPACENAME;AD\username does not exist.
Things seemed pretty messed up... I found some clues as to what was required to get things working again:
http://support.microsoft.com/kb/948679
http://support.microsoft.com/kb/823278
http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=454060&SiteID=1
According to the KnowledgeBase articles trying to create a new project while TFS is in this state may result in the following error:
TF30170: the plugin Microsoft.ProjectCreationWizard.WorkItemTracking Failed during task WITS from group WorkItemTracking
Basically Team Foundation Server has a bug that causes deleted accounts to not be resynched correctly and SharePoint doesn't handle recreated user accounts at all - you have to resynch them manually. Unfortunately in addition to this there is a bug in TFS SP1 which prevents TfsAdminUtilI from correcting the SIDS! You get the following error:
ERROR: Could not access database.
In the end I managed to correct all the problems, here are the steps required:
- Contact Microsoft Support and request the hotfix for http://support.microsoft.com/kb/934216
- Install the hotfix...
- Follow the steps in http://support.microsoft.com/kb/948679 to resynch the users SIDs in TFS
- Follow the steps in http://support.microsoft.com/kb/823278 to resynch the users in SharePoint
- Recycle the TFS App Pool to force an update of all users and groups in TFS.
Look out with users who have their workspaces mapped to "c:\documents and settings\" the new user accounts cannot necessarily access their old files there since they now log on with a new account (which just happens to have the same name).
Tuesday 15 April, 2008 (.Net | Blogging | Bugs | Fixes | Web)
I seem to have managed to get server pings from dasBlog to work in medium trust! This has been a major issue for me previously, not as much of an issue now that I primarily use off-line tools but it is still great to have it. The key is that originUrl in the medium trust settings supports regular expressions! More info on Haacked, which is where I found the solution. Just to make things clear:
<trust level="Medium" originUrl=".*" />
will allow your web site to send web requests to any external site...
Monday 04 February, 2008 (.Net | Bugs | Fixes | Web)
Recently I was working on ASP.Net 2.0 Web Parts in a Virtual PC environment and ran across a strange exception. I had read that the Web Parts personalization database is separate from the membership and profiles databases but had not really thought about it further. Previously everything just seemed to work once I ran "aspnet_regsql" to create the aspnetdb database to handle all the data for the services.
In the VPC there was already a really simple Web Part Page that I wanted to test. I had not prepared the VPC myself and others had been using it before me (so I presumed that it would work) but no one had used it in a long while. When I attempted to login and access the Web Part page I received this exception:
Procedure or Function 'aspnet_PersonalizationPerUser_GetPageSettings' expects parameter '@TimeZoneAdjustment', which was not supplied.
I checked the aspnetdb database to see if there was something wrong there but the stored procedure did not contain a '@TimeZoneAdjustment' parameter... Some related forum discussions on the web indicated that it might be a problem with the solution and database being created under a pre-release version of .Net 2.0. I tried re-running aspnet_regsql to remove and recreate the aspnetdb database, I reinstalled .Net 2.0, I installed SP1 for .Net 2. Nothing helped.
It turns out that the default personalization provider for Web Parts uses a file based SQL Express database which is automatically created in the App_Data folder of the web application. This database had already been created using a previous version of the framework before I started looking at the solution. For the sake of confusion it is also named aspnetdb, removing this database from the App_Data folder solved the problem. When I logged in and opened the Web Part page the database was recreated with the correct structure and everything just worked!
Tuesday 14 November, 2006 (.Net | AJAX | Blogging | Bugs | dasBlog | Fixes)
So far I have managed to get my dasBlog up and running. But in doing so I have disabled and broken several features:
- No HttpCompression
- No date picker while editing
- No Pingback/trackback support
It seems that I have also broken my FreeTextBox editing control. I believe that this may be an unrelated issue to the medium trust fixes and wont get into that in this post.
How could I get the broken features back up again? The strong naming issues that affect SharpZipLib and BasicDatePicker are easy to fix since there is a new free version of BasicDatePicker available BasicDatePicker Lite, the new version is not strong named and may require you to recompile dasBlog. SharpZipLib is open source so you can download it and add:
using System.Security;
[assembly:AllowPartiallyTrustedCallers]
to assemblyinfo.cs then just recompile and these functions should be up and running again, you may have to install SharpDevelop to compile the code for SharpZipLib. Any changes that were made to web.config or aspx pages to disable the functions need to be reverted and recompiled. Your web.config will need to be updated with a new version for SharpZipLib:
<
dependentAssembly>
<assemblyI entity name="ICSharpCode.SharpZipLib" publicKeyToken="1b03e6acf1164f73" culture="neutral"/>
<bindingReirect oldVersion="0.5.0.0-0.84.0.0" newVersion="0.85.0.0"/>
</dependentAssembly>
After doing this I received an "Invalid use of response filter" exception (more info at dasBlog.us) which turns out to be because my host already implements httpCompression, but disabling the blowery handler sorted it out.
TrackBack support
The main problems that remain are getting TrackBack and PingBack support. I will start by looking at getting TrackBacks working. The problem with TrackBacks is that they require dasBlog to call a web page on another site and network access to other servers is not allowed by default under medium trust so this jut won't work. I checked the TrackBack protocol to see if I could find any workarounds that could be applied. Reading the spec I realised that TrackBacks do not require the call to originate from the server that hosts the blog, if we could implement this as a client call then we could theoretically allow TrackBacks under medium trust.
I started by implementing a very simple html page to post a trackback call to localhost.
<
html><body>
<form method=post action=http://localhost/Trackback.aspx?guid=3d9f45e4... enctype="application/x-www-form-urlencoded;charset=utf-8">
<input type="hidden" name="title" value="SuperTrackback 1" />
<input type="hidden" name="excerpt" value="This is a vlient trackback call." />
<input type="hidden" name="url" value=http://localhost/Trackback.aspx?guid=dc205699... />
<input type="hidden" name="blog_name" value="TestBlogger" />
<input type=submit name="doit" value=1>
</form>
</body></html>
I have hardcoded the trackback urls just for testing. Of course it was not as easy as just posting the call since dasBlog includes spammer protection by checking the url in the trackback call. We have to disable spammer protection by checking if we can call the url at runtime in TrackBackHandler.cs (added lines are bold) .
Entry entry = dataService.GetEntry( entryId );
if ( entry != null )
{
System.Net.WebPermission urlPermission = new WebPermission(System.Net.NetworkAccess.Connect,url);
if(System.Security.SecurityManager.IsGranted(urlPermission))
{
try
{
string requestBody = null;
// see if this is a spammer
HttpWebRequest webRequest = WebRequest.Create(url) as HttpWebRequest;
webRequest.Method="GET";
webRequest.UserAgent = Utils.GetUserAgent();
...
...
}
catch
{
// trackback url is not even alive
logService.AddEvent(new EventDataItem(
EventCodes.TrackbackBlocked,
context.Request.UserHostAddress + " because the server did not return a valid response",
Utils.GetPermaLinkUrl(entryId),
url,
entry.Title
));
context.Response.StatusCode = 404;
context.Response.End();
return;
}
}
// if we've gotten this far, the trackback is real and valid
Tracking t = new Tracking();
t.PermaLink = url;
t.RefererBlogName = blog_name;
t.RefererExcerpt = excerpt;
This allows an anonymous caller to add a TrackBack to our blog but it also enables incoming TrackBacks under medium trust. This is only half of the solution for TrackBacks; we have opened up dasBlog för incoming calls to our TrackBackHandler but we still have to make calls to remote trackback urls. The next step would be to implement initiate a client-side trackback call to a remote server from inside dasBlog, this can be done in an AJAX (or is it Ajaxian) manner using XMLHttpRequest in JavaScript http://developer.mozilla.org/en/docs/AJAX:Getting_Started. I will cover this and tackling the spammer threat in future posts, since I haven't implemented a solution yet.
Tuesday 07 November, 2006 (.Net | Bugs | dasBlog | Fixes)
Since changeing web hosts I have not been able to get dasBlog running in the new environment. The host server is setup so that .Net code runs at medium trust level and won't allow me to override the trust settings. As anyone who has tried to get dasBlog to run at medium trust knows, it just isn't possible out of the box. To make life even more difficult Asp.Net returns an incomplete and ambiguous SecurityException:
Description:
The application attempted to perform an operation not allowed by the security policy. To grant this application the required permission please contact your system administrator or change the application's trust level in the configuration file.
Exception Details:
System.Security.SecurityException: Security error.
Source Error:
An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.
|
The stack trace that is included provides no help in solving the problem and actually sent me on a wild goose chase trying to solve completely unrelated issues.
I have started to dig into this to try and see if I could find a quick fix for it with googles help. So far I haven't found a quick fix but I have found a few other things that may lead me to a solution... First of all I have seen a few forum posts and blogs referring to reflection used by serialization as part of the problem. But I have managed to get XmlSerialization to work without chainging the serializer or trust level of the application so it doesn't seem to be the XmlSerializer that is causing the exception. I changed the virtual directory settings on my local machine so that dasBlog runs under .Net 2, this provided me with a much clearer exception. dasBlog has a custom XmlTextReader called XmlNamespaceUpgradeReader which inherits fom XmlTextReader. The only problem is that XmlTextReader has a security demand for exicit inheritance permissions [PermissionSet(SecurityAction.InheritanceDemand, Unrestricted=true)]. Running under medium trust you don't have the required permissions. Changing all XmlNamespaceUpgradeReader references to XmlTextReader allowed the deserialization of SiteConfig (and all other serialized data) to run.
The next problem was that even though the site serialization was working dasBlog stilled just returned a big fat exception. Another wild goose chase followed trying to solve new unrelated issues, but working on them gave me a flashback to a problem I solved a year or so ago while setting up NGallery on the same host. Calling strong named assemblies from partially trusted code can also be a show stopper if the signed assemblies don't allow partially trusted callers. Two assemblies in dasBlog cause this problem SharpZipLib which is used for HttpCompression and BasicFrame BasicDatePicker, neither are compiled with the AllowPartiallyTrutedCallers setting. Removing references to these solves the rest of the problems that I was experiencing. Unfortunately the fixes disable some functionality in dasBlog and may have unexpected effects on configurations other than mine.
I am planning on following up on this post with detailed instructions on implementing the fixes and maybe som attempts to restore full functionality.
References on the web that sort of showed me in the right direction:
http://choirulamri.or.id/archives/94
http://www.hanselman.com/blog/CommentView.aspx?guid=a7e58fa0-9390-4270-bab3-83ea2e969352
http://www.hanselman.com/blog/CommentView.aspx?guid=4dd4573a-1fee-4680-b28e-983271bd3ae8
http://www.jwerx.com/blog/CommentView,guid,31c5f741-3d52-46a3-beb5-b934dea73423.aspx
http://dasblog.us/viewtopic.php?p=761&sid=b87a704cc1fffacabcb0bfff2660aac9
Friday 14 October, 2005 (.Net | Bugs | Fixes)
There seems to be a problem in how empty DataSets handle new rows in .Net 2, we received an exception with the message "DataTable internal index is corrupted".
Our realy simple solution is to call BeginLoadData before adding the new row and then EndLoadData afterwards. This fixed the problem for us, but the actual status of the bug can be found here:
http://lab.msdn.microsoft.com/productfeedback/viewfeedback.aspx?feedbackid=ecc665a5-d71a-4e08-ac42-f3c6c5d758e0
Page 1 of 2 in the Fixes category Next Page
Disclaimer
The opinions expressed herein are my own personal opinions and do not represent
my employer's view in any way.
Sign In