After further experimentation it looks writing a WinDbg extension that will let one write WinDbg extensions in .NET is actually feasible. If you remember, I’ve talked about it in my previous post on the subject.

I could have gone the easy way and written some code in .NET exposed as a COM object and make my WinDbg extension call that managed code which will do all the rest, but that would be too easy and a lot less fun 🙂

Therefore, I’ve decide to gain some experience and knowledge in the .NET CLR hosting API.

I’m using the hosting API to bind and start the CLR, call the rest of my .NET code and pass the various interfaces that WinDbg is passing me back to the .NET code.

The challenge is to convert the interface definitions of the various WinDbg interface used to interact with WinDbg to .NET. It’s a bit of a tedious job, but someone has to do it… don’t you think?! 🙂

Anyhow, once I’ll finish the part about the CLR hosting APIs I’d write a long post about this subject and what I’ve learned while using this API so stay tuned!

Oh, one last thing. In that previous post tinkered with the idea of using some other way of instrumenting WinDbg instead of using its archaic and strange scripting language (actually that’s WinDbg’s command line interface turned into a scripting language).

Anyhow, technically I can enable any language, compile the script at runtime and run the assembly, but its a bit annoying. There is the ability of using IronPython or some other .NET dynamic language (maybe JScript .NET).

What do you think? would you like to be able to script WinDbg in your favorite .NET scripting language?

In my previous post about GC.AddMemoryPressure Tomer Gabel commented that people should not use this feature since its not the correct thing to do.

In addition to that Yuval Ararat wrote a post about this issue as well.

First of all, if I wasn’t clear enough in my first post, they are technically write BUT (and there is a big but here) there are situations, mainly in interoperability issues that might require us to better notify the GC that behind this small and insignificant .NET object lies a whole lot of unmanaged memory that is allocated and will only be released when this .NET object dies.

Calling GC.AddMemoryPressure will simply make the GC consider this object for who it really is, a memory hungry object that hides behind it a lot of unmanaged allocated memory.

Originally I just wanted to introduce this feature to people and make sure they know about it even though its quite esoteric and will mostly be used by applications with specific needs (and believe me, I know at least one application that could have benefited from this feature if it was available in .NET 1.1).

And Tomer, regarding the typos and stuff, I usually try to spell check and proof my posts, but some times they slip by 😉

Maoni has a great post about the changes and improvments of the GC in CLR 2.0.
Read all about it. Its good stuff and its Important.

In addition to that, there is a link to the slides she used in her PDC lecture.

I’m hoping to write a few more tidbits about the new and improved GC in a future post, so stay tuned.

Enjoy!

This post is a bit futuristic, but since .NET 2.0 and Visual Studio 2005 release is very near, I thought I should start to talk about it a bit more.

.NET 2.0 will introduce an improved and better GC.
One of the parameters that the GC takes into account is the amount of memory a certain class instance takes. By doing so, the GC can better understand how much memory will be gained by collecting this object.

One of the inputs the GC takes into accoutn when decided whether to initiate a collection or not is the amoutn of managed memory allocated.
If we have a managed class instance that doesn’t allocate a lot of mamanged memory but holds a pointer to a large unmanaged memory (either a reference to a COM object that allocates a lot of information, or directly allocating unmanaged memory using functions such as Marshal.AllocHGlobal) the GC will not know about the unmanaged memory allocated and will not consider scheduling a GC sooner.

This means, that if there is no reference to that object and its finalizer releases the unmanaged memory, until there is a GC and the finalizer thread reaches that object, this unmanaged memory will not be released and may add pressure on the application’s memory usage.

For this purpose, in .NET 2.0 the “AddMemoryPressure” function was added to the GC class.

“AddMemoryPressure” allows the developer to notify the GC about the amount of additional unmanaged memory that was allocated in different places in the application. The GC will take this into account when considering the schedualing of a collection.

For example, if at some point in the application I create a COM object that I know allocates a bit chunk of memory, after creating it I will call “AddMemoryPressure” and give it a rough estimate of the amount of unmanaged memory this COM object takes.

For example:

class MySpecialBitmapClass
{
private long size;

MySpecialBitmapClass(string fileName)
{
size = new FileInfo(fileName).Length;
GC.AddMemoryPressure(size);
}

~MySpecialBitmapClass()
{
GC.RemoveMemoryPressure(size);
}
}

When I create the class I say that I will add a memory pressure which is at least as large as the file I’m working on. When the instance of this class is being finalized it will remove the pressure.

Be sure to tune in to hear some more changes and updates that are coming in .NET 2.0’s GC.

.NET allows creating Windows Services which are commonly used for unattended services such as Remoting containers.

For some reason, when using a Windows Services as a container for your Remoting application or as just a Windows Service that perfrom various tasks, the GC used is the workstation GC.

We have previously talked a bit about the difference between the workstation GC and the server GC but I’ll explain a bit about them again.

Workstation GC
The workstation GC is, as its name applies, is used in a workstation scenario. It is optimized for single CPU machines and for desktop application by using the main thread to perform the GC.
It uses 16mb segments that it reserves and sub allocates.

It has an option called “Concurrent GC” which allows the GC to run on a dedicated thread.

Server GC
The Server GC It is optimized for server operations and works only on a multi processor machines (CPUs that has Hyper Threading enabled are considered as two CPUs).

It has a GC heap per CPU and a thread per CPU that performs the garbage collection.
It uses 32Mb segments that it reserves and sub allocates.

All of these features make the Server GC more appropriate for demanding server applications due to its higher throughput.

Who uses the Server GC?
The only containers that use the Server GC by default are ASP.NET and COM+ (through Enterprise Services).

All other applications including Windows Services use the Workstation GC by default.

This means that even if you wrote a cool Windows Service in .NET that does cool stuff it may suffer from using a non optimized GC even though its a high throughput service that serves millions of users.

So, what can we do about it?
Before .NET Framework 1.1 SP1 you had to implement your own container for the CLR.
Since .NET Framework 1.1 SP1 you can just add to your app.config file the following tag and it will tell the GC to use the Server GC (of course, only if you have more than one CPU):

<configuration>
<runtime>
<gcserver enabled="true" />
</runtime>
</configuration>

You can read more about it (though not too much) in this Microsoft KB article.

For .NET Framework 1.0 you’ll still have to implement your own container for the CLR.
There are a bunch of these hangging around. A nice one is this one which is posted in The Code Project.


Just think about the fact that with one simple flag you can boost your Windows Service performance.

I was working today on figuring out some Managed C++ code with a friend from work (Oran Singer) and I just wanted to outline some of the oddities of the Managed C++ Destructors because I think it is important for everyone to understand how they work and what some of the quirks about them are.

We are still in the middle of figuring out something more serious (I will post on that later on).

I would like to “appologize” to the C#/VB.NET crowed in advance since this post is all about Managed C++ today a bit on its future, but I think it will be interesting none-the-less.

First of all, a Managed C++ class that had a destructor will include two additional methods due to the destructor:

  1. Finalize method
  2. __dtor method

Currently you cannot implement the finalize method in managed C++ directly like in C# by simply override it since it is defined as private protected in System::Object so the only way to do it is to add a destructor to the class.

This means that a Managed C++ class that has a destructor has a finalizer with all the implications of a .NET finalizer (I’ve written in more detail about finalizers here).

The __dtor is being called when you use the “delete” operator on a managed C++ class instance (IMPORTANT NOTE: You can only call “delete” on a managed C++ class instance that implements a destructor).

The implementation of the __dtor method is similar to an implementation one would put in a Dispose method when implementing the IDisposable interface. It has two function calls, one to GC.SuppressFinalize (so the finalizer thread will ignore calling the Finalize method of this instance) and a call to Finalize (to actually perform the destructor code).

Another interesting anecdote is that if you have a pure virtual class and you add a constructor to it, it must have a body even though this is a pure virtual class since there is always a virtual call to the base class’s destructor.

Looking a bit to the future, in Visual Studio 2005 (previously known as Whidbey) Visual C++ that generates managed code get a few new features that are a bit different in implementation than what we saw above.

First of all, when implementing only a destructor (“~MyClass()”) the compiler will actually fully implement the IDisposable interface and in certain cases will automatically call the Dispose method (in C# you would either use try..finally block and call the Dispose at the finally part or use the “using” keyword) in the following cases:

  • A stack based object goes out of scope
  • A class member’s enclosing object is destroyed
  • A delete is performed on the pointer or handle.

In addition to that you can explicitly implement a finalizer using the “!” tag (“!MyClass()”). That finalizer is subject to that usual finalizer rules we already know. The main advantage is that it will NOT run if you already called the destructor.

These new features allows improved control for a more deterministic cleanup and allows finer control on C++ managed objects that was previously not available.

I would like to thank Oran again for his help, mainly for forcing me to search some more on the subject and getting the idea to write this post.

Remeber a few posts ago that I talked about CorDbg, the managed debugger?
According to this post in Mike Stall’s blog (a fine blog, btw) its out of the SDK.

CorDbg was the first managed debugger that was written. It was written in native code since Interop in those days was still being written and was very unstable.
CorDbg also lacks features that are common in WinDbg such as extending it with extensions and instrumeting with it in a nice way.

Its hard (or even impossible according to this at Mike Stall’s blog) to implement a managed debugger in managed code for v1.1 (although I know there are a few people messing around with it as we speak, me included 😉 ). That is why CorDbg is the only viable managed debugger for production debugging for v1.1

To fill all of these gaps and to make things a lot more easier and more powerful here comes MDbg (ta-da).

MDbg is a managed deubugger fully written in C#. Its extendable by writing managed code to extend it, can be used to instrument code and is just so damn cool 😉

It gives you all the things you ever wanted from a production debugger and can also debug both .NET 1.1 and 2.0 code (to use it to debug 1.1 code you will still have to have the .NET 2.0 Runtime installed).

You can find out more information about MDbg here.
I am in the process of collecting some more information for a more elaborate post on MDbg and some of its features (probably a series of posts), so stay tuned.

In this post, I’ll talk about some issues that one must understand about mixing finalization and P/Invoke Interoping.

Usually, when you have a class that has an IntPtr as a member due to some P/Invoke code you implement a Finalizer and the IDisposable interface in which the necessary cleanup code will be implemented to release the unmanaged resources you have allocated.

The main problem starts when that same class has other member variables that are managed.
These managed member variables will not get collected until our class’ instance is finalized and that can generate memory pressure that is not really necessary if things are implemented correctly.

So, what can we do about it?

There are a few approaches that can help us minimize the effect of P/Invoking and finalizers:

  1. Don’t declare the IntPtr variable as a member variable.
  2. Create a class that wraps the IntPtr and P/Invoke code without declaring managed member variables.

Let’s examine both approaches.

The first approach talks about not declaring IntPtr variables as member variables. This means that if you have some function that needs to interop, delcare the IntPtr variable as a local variable, use it and clean everything up. This is the best approach but will only work if there are no (serious) performance penalties such as calling some heavy P/Invoked Windows API.

The second approach is a more viable one if there is a need to continuously call some unmanaged API such as ReadFile on a socket or file. In this case, writing a dedicated class that holds the reference to the unmanaged resource only and a finalizer and IDisposable would be a great way of making sure that it will not hold back the GC of other resources (not to mention it would be cleaner to have a dedicated class for wrapping such API 😉 ).

That class we have an IntPtr as a member variable and will have additional functions to manipulate the data that is being held behind that unmanaged resource.

By implementing one of the two approaches I’ve discussed above you can, quite easily, ease the memory pressure on your application with little change to your code and functionality.

In .NET Framework 2.0 that will come with Visual Studio 2005 (formerly known as Whideby) there is a new mechanism to handle such issues called SafeHandle. You can read more about it here, in this great post from the BCL Team Blog.

I will talk about SafeHandles in a subsequent post.

WinDbg is a native debugger and although using it combined with SOS to perform managed debugging is great, it sometimes lack the finesse that only a managed debugger can give.

CorDbg is a managed debugger that ships with the .NET Framework SDK. It was not written in managed code since it was the first managed debugger written in the early days of .NET when you couldn’t yet rely on managed code to produce a debugger that will debug it.

CorDbg knows and understand .NET better than WinDbg+SOS can. That is why you can better debug .NET applications with it.
For example, A simple scenario that is hard to do with WinDbg+SOS and very easy to do in CorDbg is to break only when specific .NET exceptions happen.
When I say specific I mean SPECIFIC, i.e., I can specifically specify that I want to ignore System.ThreadAbortException but not System.FileNotFoundException.

To do that all I need to do is run the following commands:

catch exception System.FileNotFoundException
ignore exception System.ThreadAbortException

WinDbg can only break (without adding sophisticated scripts to further analyze an exception) on CLR exceptions since all CLR exception has the same exception code.

Some more cool and useful things CorDbg can do includes:

  • Easily set a breakpoint in managed code (and we all know its not that nice to set a managed breakpoint in WinDbg using SOS) using the b[reak] command:b foo.cs:42 – will break in file foo.cs in line 42.
    b MyClass::MyFunction – will break at the start of MyClass::MyFunction
  • Disassembling to native or IL code using the dis[assemble] command.
  • Evaluate a function and using the f[unceval] command:f MyNamespace.MyClass::MyMethod – for static method
    f MyClass::MyMethod param1 param2 $result – for instance method. The result is stored in $result for further evaluations
  • Printing variables to see their content using their names (for local variables, only if debug symbols are available) using the p[rint] command instead of finding out their addresses to print them in WinDbg:
    p obj1.var1 – for variables inside objects
    p MyClass::MyStaticVar – for static variables
    p arr[2] – for print a specific cell in an array
  • Changing a variable value using the set command.
  • Setting the next line of execution by simply invoking setip [line number] (like you do with right-clicking in Visual Studio .NET)

As you can see, using a managed debugger like CorDbg can be a lot nicer and easier instead of WinDbg.

I heared unsubstantiated rumores that when Microsoft developers have problems with a certain build of Visual Studio they use CorDbg to figure out the problem or to debug instead of finding what the problem with Visual Studio is.

Visual Studio .NET 2005 will contain a new managed debugger that was written in managed code called MDbg. You can get it here (it required .NET Framework 2.0). MDbg has an extensions API so you can extened it just as you can extened WinDbg (but its even easier). It also features a nice simple GUI for those of you who doesn’t like to use the keyboard too much.

You can read more about MDbg in Mike Stall’s blog

I will dedicate another post to MDbg so stay tuned.

Yesterday my RSS client, RSSBandit, decided to stop working.
Luckly RSSBandit is a .NET application with source code, so I’ve decided to start debugging it.
It kept on failing when it tried to deserialize one of its configuration classes from Xml using the XmlSerializer class.

The XmlSerializer class generates a temporary assembly in its constructor according to the XmlMapping it receives, compiles it and loads it.
Sometimes there is a problem at compile time. The code is not actually compiled and you will get a FileNotFoundException exception just for calling “new XmlSerializer(typeof(MyClass))“.

While trying to resolve this issue I’ve stumbled upon two tools that can help you better understand the cause of the problem and at some scenarios might suggest a resolution.

The first tools is a command line too called XmlSerializerPreCompiler written by Chris Sells. You just specify the full path to the assembly and the type name to check and it will try to create an XmlSerializer passing the type to it and analyzing the error.

The second one is a GUI tool that was based on XmlSerializerPreCompiler called XmlPreCompiler written by Mathew Nolton.

Both tools comes with full source code.

While I didn’t have time to check their source code I did see one of the things they use to get more information about the temporary assembly that is being compiled on the fly when creating the XmlSerializer.
It seems that there is a configuration flag that you put in the app.config file (if you don’t have one, just create one that is named myprogram.exe.config, where myprogram is your executeable name).

Just add the following to your app.config file:

<configuration>

<system.diagnostics>

<switches>

<add value=”4″ name=”XmlSerialization.Compilation”>

</switches>

</system.diagnostics>

</configuration>
This option tells the XmlSerializer not to delete the temporary files created to generate the temp assembly. It will leave the .cs file it was trying to compile, a .cmdline file which has the command line parameters that are sent to the compiler and a .out file which has the full command line that was used to compile the .cs file.

All of these files are generated it your %USERPROFILE%\Local Settings\Temp folder (which is usually equivalent to %TEMP%) and by just adding the above flag in your app.config you can get the input that is used in the programs I’ve mentioned above and investigate on your own.

While all of these programs couldn’t help me in resolving what is wrong with RSSBandit (and for that matter, any thing that has the “new XmlSerializer(typeof(MyClass))” code in it that runs on my machine) I’d thought it would be nice to share this information with everyone.

Since I don’t have time to further analyze this and I need my daily RSS feed dose, I did the following things:

  1. Ran WinDbg and started RSSBandit through it.
  2. Set a breakpoint (we have discussed about setting a managed breakpoint before) on System.CodeDom.Compiler.CompilerResults.get_CompiledAssembly (this function is the one that tries to load the temp compiled assembly and will throw the exception I’m getting in RSSBandit since the temp assembly wasn’t compiled).
  3. When the breakpoint hit (and it does so a few times until RSSBandit finishes loading) I ran !dumpheap -type CompilerResults
  4. I dumped the CompilerResults object using !dumpobject
  5. I then took the address of the pathToAssembly member and used !dumpobject on it as well.
  6. I then took the path, went there, opened the file named [strange random chars].out (where [strange random chars] is a unique name for each such assembly), opened it, copied the command line and ran it to generate the assembly.
  7. Continued doing this until we never reach the breakpoint again.

What I did learn from this workaround is that the main problem is executing the compiler command to compile the assembly and that will be the point from which I will continue to troubleshoot this issue.

At least for now RSSBandit is working (though only through WinDbg 😉 ) and I can read my favorite RSS feeds.