I haven’t written in a while and I don’t like being an echo blog but this couldn’t pass right along.

ScottGu announced 2 days ago (yeah, I’m slow to catch that up, but its been a holiday here in Israel 😉 ) that they’ll release the full source code of the .NET framework library with Visual Studio 2008 so that everyone can browse it as well as be able to debug (which is always helpful).

In addition to that, it seems that VS 2008 will have integrated symbols server support which also includes downloading the debug symbols as well as source code directly from Microsoft.

Way to go Microsoft. This is exactly what I was missing for quite a long time!

Sometimes in ASP.NET when you are trying to access a certain page you might get the following error:

External component has thrown an exception

In some other situations in .NET you might get this error as well (not directly related to ASP.NET, but sometimes to Interoping code or COM code), but this is beyond the scope of this post.

In the cases that I’ve stumbled upon, the reason is usually related to on-the-fly compilations ASP.NET performs when first accessing a certain page on an application.

The main problem with this exception that it is less than informative 🙂 . The only way of actually figuring out what is wrong (if this is indeed a case related to compilation error like I’m talking about) is to dig deep using WinDbg.

Luckily, every compilation that runs inside ASP.NET uses an API that upon encountering an error will create an object in the heap called CompilerError.

To debug this issue perform the following steps:

  1. Attach WinDbg to the ASP.NET worker process (w3wp.exe in Windows 2003, aspnet_wp.exe in Windows XP/2000)
  2. Make sure you have CLR exceptions enabled (see this previous post about how to enable CLR exceptions in WinDbg).
  3. Try to access the page that is not working, it should throw an exception and WinDbg should break in.
  4. Run the following command:

    !dumpheap -type CompilerError

    This should produce a list of all CompilerError objects which should allow us to inspect them and see the exact compilation error.

  5. Now dump each of the objects using the following command:

    !dumpobject 0xXXXX

    Where 0xXXXX is the address of the objects listed in the list you got from the !dumpheap command.

  6. At that point you should see the CompilerError object itself and you can dump its errorMessageText member (it’s address is near its name) using !dumpobject which should show you the exact compilation error.

In my case, the problem was that in the web.config file under the <compilation> / <assemblies> element there was no reference to one of the AJAX.NET assemblies (System.Web.Extensions) which caused the page to fail during on-the-fly compilation.

Do remember that this exception sometimes has nothing to do with compilation. If you don’t have any object of type CompilerError or there are a few but they make no sense in regards to the page you are accessing it might be something else.

While I don’t normally repost links and try to avoid it as much as possible, I’ve stumbled upon this post by Jeff Atwood (he works on CPU-based parallelism in the .NET CLR team sorry I mixed things up a bit. Jeff Atwood doesn’t work on the CLR team and not for Microsoft. David, thanks for pointing that out to me. I guess I just mixed things up a bit, since I also wanted to place a link to Joe Duffy’s post about thread priorities) about Thread Priorities.

Read it carefully and think twice before writing:

thread.Priority = ThreadPriority.BelowNormal

While the sample Jeff gives is for CPU starvation and the problem he is illustrating will likely be caused on machines with a single CPU, it IS important to understand that thread priorities WILL affect your application, sometimes in the worst possible way.

While I’m at it (“it” being doing a links post 🙂 ), Maoni (which got a GREAT blog about .NET and its internals) has a philosophical post about performance work. This post really highlights the fact that general fundemental knowledge is always useful when trying to understand a problem.

If pre-optimization is the root of all evil, fixing things that are not there or are not the problem is the fertizlier that makes the root of all evil grow and grow and grow, consuming your code!

While I know its been quite here due to my “other job” (which I highly recommend you check 😉 ), I have a few things down the pipe that will, hopefully, be of use to all of you.

A friend of mine asked me a question which reminded me a very important issue regarding STA (Single Threaded Apartment) COM objects in ASP.NET Web applications. STA COM objects were created so that most people won’t have to deal with threading issues. The COM framework itself will make sure that each call to any method in an STA COM object will be blocked so that other requests from other threads won’t be able to get into that certain method until a previous call exited it (they will be queued).

My friend tried to create an STA COM object within the web application. The COM object was suppose to access a network share on some other machine in the network.

The ASP.NET process was configured to use the default IUSR_XXX (where XXX is your machine name) user, which has little access to anything.

In order to access the network share we had to change the context of the calling thread, so my friend used the <identity> tag in the web.config to make sure each request to the page running this code will impersonate to the user specified in the <identity> tag.

The problem starts with the threading model of ASP.NET. ASP.NET threads which executes the various request it receives are MTA (Multi Threaded Apartment). When an STA COM object is created on an MTA thread, to actually being able to perform the synchronization that STA COM objects give, we will have to context switch to the default STA thread of the process (which is a dedicated thread for that purpose) and the COM object will be created there.

Here our problem starts. The <identity> tag makes sure that the thread executing the request (the MTA thread) will impersonate its security context to the user specified in the <identity> tag but our STA COM object eventually was created on the default STA thread which was not impersonate, causing it to get the security context of the process (which was IUSR_XXX – the least powerful user of all). This meant that when the STA COM object tried to access the network share it failed since it didn’t get the right credentials.

So, how can we solve this issue? There are a number of ways.

The easiest way of all is to change the user running the ASP.NET application. To do that in Windows XP or Windows 2000 you need to go to the machine.config file and change the username and password in the processModel section. It is not recommended to put your password in clear text in the machine.config. That is why Microsoft released this utility which will encrypt your username and password in the registry.

In Windows 2003, you’ll need to create a new Application Pool and make sure the identity of that pool is the needed user that can do all the necessary things, security wise, that your application needs to do.

Another option, is to use the identity flag as specified in MSDN but to add the ASPCompat attribute to the your page class (or enable it in the ASPX page as described here). This will make sure that each of your requests to that page will actually run in an STA thread instead of an MTA thread, so when creating an STA object from within this page, the object will be created on the calling thread (since we are already in an STA thread) which will have the correct identity (we are using the <identity> configuration in the web.config).

And the last option, which works in .NET Framework 1.1 SP1 and above and in .NET Framework 2.0 (though I haven’t tested it in .NET 2.0), is to add a special flag called “apartmentThreading” which essentially changes all of your page to work the same way as adding the ASPCompat attribute.

In any case, in order not to hurt performance and scalability too much, use as little STA COM objects as you can and specifically mark the pages calling these STA objects using the ASPCompat attribute. When you have to use an STA COM object, create it, use it and released it as soon as you can, preferably using Marshal.ReleaseCOMObject (which will force it to die as soon as you call this method instead of waiting for the next garbage collection. To get more information about what Marshal.ReleaseCOMObject does see my previous post).

Update 13/6/2006: Updated a few typos.

At my day job (which is part of the reason I’m posting less frequently) I’ve had to P/Invoke a bit and gathered some P/Invoke tips that I’ve decided to share.
Today’s tip is about P/Invoking and C style unions.

Let’s take the following union:

union MYUNION {
int number;
double d;
}

This is quite a standard union, which means its memory is contiguous. Its translation in C# would look like this:

[StructLayout(LayoutKind.Explicit)]
public struct MYUNION {
[FieldOffset(0)]
public int number;
[FieldOffset(0)]
public double d;
}

Notice two important things. First, the StructLayout attribute is set to LayoutKind.Explicit, this means that using the FieldOffset attributes we are building the memory layout of the struct and it will look exactly like this. When usually handling C structs in .NET and P/Invoking we usually use LayoutKind.Sequential so that the memory layout will be as described using the layout of the fields as we have put them.

The second important thing is the FieldOffset attribute. Notice that both fields have the same offset. This will tell the P/Invoke layer that these two fields are actually part of the same memory (remember, these two fields were declared inside a union!).

This is how unions will work for you in your P/Invoke code. Enjoy!

Technorati : , , , , , , ,
Del.icio.us : , , , , , , ,
Ice Rocket : , , , , , , ,

A friend of mine that worked with me on my previous work place talked to me the other day and asked me why we put all of our assemblies in the GAC. After all, it does make a bit of a mess during the installation and no one actually see the benefit of it since a lot of the .NET first hype was “X-Copy Installation” and the “End to DLL Hell”.

As it usually goes in these situations, the “End of DLL Hell” brought its own set of challenges to overcome, therefore, I’ve decide to dedicate this post to the GAC and explain what is it for and how and when to use it.

What is the GAC?
So, what is the GAC? It stands for Global Assembly Cache and has two major functions:

  1. Provides a place to put a few versions of the same assembly that will co-exist side by side.
  2. Provides a place where one can place the native image of an assembly so that it will save us the need to perform Just In Time compilation (JIT) on the fly the first time we call to a certain method in one of the objects in that assembly.

In the old days before .NET and COM people relied on the search path to strategically place DLLs so that the application will find them and load them. There was a certain order and logic to the search path and people usually tried to go with it. If you wanted to place a DLL that will be accessible to all without changing the PATH environment variable, you usually put it in the SYSTEM directory (and later in the SYSTEM32 directory).

This process usually caused the DLL Hell since the application relied on a correct PATH envrionment variable and if, for some reason, it was changed or the same DLL in a different version was in a place before your DLL, your application would load the wrong version and bad things would start the happen.
At best, you can’t see some bug fixes and a new functionality, at the worst case, your application just crashes and strange places (depending how the application is written).

After the old days, came what I’d like to call the “Middle ages” and the era of COM which relied a bit on the search path as in the old days, but now it was written in the registry where to actually look for the exact DLL. If it couldn’t find it, it would rollback to the old days and usea the PATH environment variable and the search logic to try and find the DLL.

In the .NET era the GAC first major function comes to our aid. No more multiple places to put DLLs. We now have a single place that can provide a place to all of the versions of our assemblies all together. We can specifically redirect application that are suppose to use a certain version of the assembly to a different one using policy files and if everything is indeed in the GAC we are no longer in the mercy of the dreaded search path (although, if we can’t find the assembly in the GAC, the loader will revert to the old search path ways, as explained in this post).

The second major function of the GAC is to host the native images of assemblies that were compiled using the NGEN tool. If you wish to read more about NGEN and when to use it I suggest this fine post from Jason Zander‘s blog.

How to use the GAC?
The first basic condition of putting an assembly into the GAC is to strongly sign it. How to sign an assembly is beyond the scope of this post (but I will write about it in a future post).

After the assembly is strongly signed, you simply drop it in the “Assembly” folder located under you Windows Home Directory\assembly (that’s where the GAC is actually stored).

That’s it.

When to use the GAC?
OK, so now that we know how to use the GAC and what it was meant for, when should you use it?

Well, the rule of thumb is this… If you have a shared component that is used in more than one application OR your application exposes some kind of an API that can be used by other, you should definitely put your assemblies in the GAC. After all, this is one of its major uses!

If you have a simple executable or a simple application that only uses the assemblies in its own directory there is no need in putting anything in the GAC.

Quite simple, isn’t it?

In the next post I’ll dig a bit more into the inner workings of the GAC so stay tuned!

I’m often being asked to resolve various problems in which the .NET Assembly loader does not find the right assembly to load and therefore doesn’t load it at all.

While this is not hard core debugging that you would usually expect using WinDbg, CorDbg, MDbg and others, this is a very common case that with a little more information about how .NET works and one simple tool, can be easily resolved in both development and production environments.

Did you see the error “File or assembly name XXX, or one of its dependencies, was not found” error (replace XXX with the relevant assembly name) before?

That’s exactly the error I’m talking about.

Before we will go into common problems and how to resolve them, let us first understand how the .NET assembly loader actually works and how and where it searches for the assembly we want to load.

Strongly Signed Assembly
A strongly signed assembly is an assembly that was signed using a key pair during compilation (or after it).
The key pair is generated using the “sn.exe” utility and can be unique per assembly, per product or per publisher (depending on your needs).

Strong signing an assembly gives the assembly a unique identifer which is composed out of the following parameters:

  1. Assembly Name – the file name of the assembly minus the “.dll” suffix.
  2. Assembly Version – The version of the assembly as defined in the AssemblyInfo file. It can contain a fixed number which the developer / build machine will advance manually or have the “*” sign at the end so each build will increment it automatically.
  3. Culture – The deafult value for this parameter in most assembly is “neutral” meaning, that this assembly is not culture specific. It is mainly used to create assembly (or satellite assemblies) that are culture specific such as a specific assembly which has resource string that contains German text instead of English text. In which case, the culture will be “de”.
  4. Public Key Token – A token generate from the public key that you have in your key pair. It is used to verify the signature of the assembly to check that it was not tempered with after it was signed.

All of these parameters together create, what is referred to as, a “Fully Qualified Name” of a .NET assembly and looks like this:

MyAssembly, Version=1.0.2.3, Culture=neutral, PublicKeyToken=b77a5c561934d069

A fully qualified name for a specific type will look like this:

MyType, MyAssembly, Version=1.0.2.3, Culture=neutral, PublicKeyToken=b77a5c561934d069

If there is a program that needs to load some type and create it dynamically and the assembly of that type is strongly signed you can use the following piece of code to load and create that specific type given its fully qualified name:

string typeFullQualifiedName = “fully qualified name of type here”;
Type type = Type.GetType(typeFullQualifiedName);
if (type != null) {
object obj = Activator.CreateInstance(type);
}

This piece of code will try to load the relevant assembly (we will talk about the .NET assembly loader logic next) and if it succeeds, it will get the Type object which will allow us to create an instance of that type.

If you don’t have a full qualified name of the type, you can always use System.Reflection.Assembly.Load, give it the full path (or relative path) to the assembly and call assembly.GetType with the type you want to get and continue from there on.

.NET Assembly Loader – The real story

The first thing the .NET assembly loader checks is whether the assembly is strongly signed. If it is, it will start its search in the Global Assembly Cache (a.k.a the GAC. You can read more about it here).

The loader will search for a policy file named in the format of:
policy.AssemblyMajorVersion.AssembyMinorVersion.AssemblyName

For example:
Policy.1.2.MyAssembly

If such a file exists it will look inside of it if the version of the assembly that we are trying to load matches the version/versions range written in the policy file. If it does, it will try to load the assembly with the version specified there. If no such policy file exists, it will try to load the assembly from the GAC.

If it will fail to find it in the GAC, it will start to serach in the system’s search path.
In web applications it will also include the application’s Bin directory in the search path.

You can manully add folders to an AppDomain’s search path by using the “AppendPrivatePath” method.

So, what should I do if it can’t find the assembly?
The best method I’ve found to check if there is a policy problem or a path/version problem is to use FileMon from SysInternals.

FileMon tracks all activities that relates to accessing files in your operating system including read, write, access denieds on files and so on.

I would suggest adding a filter with the name (or partial name) of your application and start the application.

It will start showing you all file access that your appliction does. When you’ll receive the assembly load error, search for “NOT FOUND” or “ACCESS DENIED”.

If you have policies for that specified assembly you will see that the application accessed the policy file and directly after it will try to load the assembly with the version specified in the policy file.

If this is not a strongly signed assembly or the file is not in the GAC you will see all the places the application searched for to load the file.

Summary
This is a very easy trick that saves long hours of debugging, specifically in a production or semi-production environment.

If you are using the various abilities of .NET for side-by-side versioning, policy files and the GAC, do mind that you WILL run into these simple problems that consumes a lot of time and effort.

If this post helped you, comment on this post or just Email me. I would like to know that I’m not the only one fighting with the GAC and policy files 🙂

In my previous post about GC.AddMemoryPressure Tomer Gabel commented that people should not use this feature since its not the correct thing to do.

In addition to that Yuval Ararat wrote a post about this issue as well.

First of all, if I wasn’t clear enough in my first post, they are technically write BUT (and there is a big but here) there are situations, mainly in interoperability issues that might require us to better notify the GC that behind this small and insignificant .NET object lies a whole lot of unmanaged memory that is allocated and will only be released when this .NET object dies.

Calling GC.AddMemoryPressure will simply make the GC consider this object for who it really is, a memory hungry object that hides behind it a lot of unmanaged allocated memory.

Originally I just wanted to introduce this feature to people and make sure they know about it even though its quite esoteric and will mostly be used by applications with specific needs (and believe me, I know at least one application that could have benefited from this feature if it was available in .NET 1.1).

And Tomer, regarding the typos and stuff, I usually try to spell check and proof my posts, but some times they slip by 😉

Maoni has a great post about the changes and improvments of the GC in CLR 2.0.
Read all about it. Its good stuff and its Important.

In addition to that, there is a link to the slides she used in her PDC lecture.

I’m hoping to write a few more tidbits about the new and improved GC in a future post, so stay tuned.

Enjoy!

This post is a bit futuristic, but since .NET 2.0 and Visual Studio 2005 release is very near, I thought I should start to talk about it a bit more.

.NET 2.0 will introduce an improved and better GC.
One of the parameters that the GC takes into account is the amount of memory a certain class instance takes. By doing so, the GC can better understand how much memory will be gained by collecting this object.

One of the inputs the GC takes into accoutn when decided whether to initiate a collection or not is the amoutn of managed memory allocated.
If we have a managed class instance that doesn’t allocate a lot of mamanged memory but holds a pointer to a large unmanaged memory (either a reference to a COM object that allocates a lot of information, or directly allocating unmanaged memory using functions such as Marshal.AllocHGlobal) the GC will not know about the unmanaged memory allocated and will not consider scheduling a GC sooner.

This means, that if there is no reference to that object and its finalizer releases the unmanaged memory, until there is a GC and the finalizer thread reaches that object, this unmanaged memory will not be released and may add pressure on the application’s memory usage.

For this purpose, in .NET 2.0 the “AddMemoryPressure” function was added to the GC class.

“AddMemoryPressure” allows the developer to notify the GC about the amount of additional unmanaged memory that was allocated in different places in the application. The GC will take this into account when considering the schedualing of a collection.

For example, if at some point in the application I create a COM object that I know allocates a bit chunk of memory, after creating it I will call “AddMemoryPressure” and give it a rough estimate of the amount of unmanaged memory this COM object takes.

For example:

class MySpecialBitmapClass
{
private long size;

MySpecialBitmapClass(string fileName)
{
size = new FileInfo(fileName).Length;
GC.AddMemoryPressure(size);
}

~MySpecialBitmapClass()
{
GC.RemoveMemoryPressure(size);
}
}

When I create the class I say that I will add a memory pressure which is at least as large as the file I’m working on. When the instance of this class is being finalized it will remove the pressure.

Be sure to tune in to hear some more changes and updates that are coming in .NET 2.0’s GC.