Martin's Activity Log

Booting SuSE 11 inside VMware

Two weeks ago, I got a new laptop and set it up to dual-boot both Windows Vista (which I need for work) and SuSE 11. Inside Vista, I also installed VMware and made it boot my existing Linux installation.

Using raw disk partitions in VMware is a bit dangerous and you have to be really careful when setting this up. Here's how I configured it - and it turned out to work really great:

• First of all, make sure you select individual partitions in VMware and not the entire disk. Make sure you don't select the partition containing your host Windows operating system. This ensures that VMware doesn't let the guest operating system access that partition, so you can't accidentally boot or mount it - which'd result in severe data corruption and possible loss of all data on the disk.

• Before booting the first time inside VMware, you need to enable SCPM and create two profiles - I'm using gondor (running natively) and isengard (running inside VMware). In yast, go to System / Profile Manager, active it there and make sure to set both Switch Mode and Boot Mode to Save Changes.

  Then do

  scpm copy default gondor
  scpm switch gondor
  scpm save
  

  on the command line. This'll save your current configuration into the gondor profile. After that, do a

  scpm copy gondor isengard
  scpm switch isengard
  scpm save
  

• Edit /boot/grub/menu.list, replace the SuSE 11 with Gondor, add PROFILE=gondor to the kernel command line and make sure you're using root=/dev/disk/by-uuid:

  title Gondor
      root (hd0,2)
      kernel /boot/vmlinuz root=/dev/disk/by-uuid/b39b72d4-ac4e-4385-b9a3-08ca81b4638f splash=silent PROFILE=gondor showopts vga=791
      initrd /boot/initrd
  
  title Mordor
      rootnoverify (hd0,2)
      chainloader (hd0,0)+1
  

• Copy /boot/grub/stage2 to /boot/grub/state2-isgengard and also copy /boot/grub/menu.list to /boot/grub/menu-isengard.lst. Edit /boot/grub/menu-isengard.lst, remove the Mordor entry and replace Gondor with isengard. This configuration will be used when booting inside VMware:

  title Isengard
      root (hd1,2)
      kernel /boot/vmlinuz root=/dev/disk/by-uuid/b39b72d4-ac4e-4385-b9a3-08ca81b4638f splash=silent showopts PROFILE=isengard
      initrd /boot/initrd
  

• Edit /etc/fstab and use /dev/disk/by-uuid everywhere instead of /dev/disk/by-id or using the raw device. My /etc/fstab looks like this:

  /dev/disk/by-uuid/b39b72d4-ac4e-4385-b9a3-08ca81b4638f /                    ext3       acl,user_xattr        1 1
  /dev/mapper/cr_sda6  /home                ext3       acl,user_xattr,noauto 0 0
  /dev/disk/by-uuid/13aee839-dccc-4999-a977-6168d109dae1 /work                ext3       acl,user_xattr        1 2
  

  This is important since your hard disk will have different physical devices when running inside VMware, so /dev/disk/by-id will be different.

• Now you can reboot your system into Windows.

• In VMware, create a really small (10 MB is fine) virtual disk in addition to your physical hard disk. Double check that you're using individual partitions and the partition containing your Windows host operating system is not selected.

• Boot inside VMware, press F2 to go to BIOS setup, select the virtual hard disk as primary boot device, save and exit. Press Esc to go to the boot menu, select CD-ROM as temporary boot device and boot off the SuSE 11 installation DVD, select Rescue System

• Once in the rescue system, /dev/hda should be the virtual hard disk and /dev/hdb your physical one. Manually install grub into the MBR of the virtual disk and make it use /boot/grub/stage2-isengard and /boot/grub/menu-isengard.lst. You have to do that inside the grub shell, by using appropriate root and install commands - this is a bit dangerous since it could make your system unbootable, so check the manual.

  Note that we only need the MBR in the virtual hard disk - the boot loader in it will load /boot/grub/stage2-isengard from your physical hard disk. Grub encodes the path of the configuration file (/boot/grub/menu-isengard.lst) in the stage2, that's why we're using a custom one.

  This setup is a bit complicated, but it ensures that you don't accidentally boot Linux with the wrong profile.

• Remove the SuSE 11 installation DVD and reboot.

With the two different boot loaders, you'll always boot your Linux with the correct SCPM profile. The SCPM profiles make sure you can have different configurations when running natively and inside VMware. For instance, your /etc/X11/xorg.conf will be different in each scenario and you may also have different network settings.

I'm even using different host names and different IP addresses for both configurations - the one "machine" is called gondor.trier.ximian.com (192.168.3.8) and the other one is called isengard.trier.ximian.com (192.168.3.9). You may also add additional files / services to SCPM to also save things like /etc/issue and /etc/motd, for instance.

SCPM also takes care of the different X11 configuration - when running natively, I'm using a multi-headed setup with an external monitor - and in Vista, I move the VMware window into the external monitor and use full-screen mode there. This means that I have Vista on my laptop's main display and my GNOME session on the external monitor - really awesome !

On Isengard, I'm also using Samba to mount /work/mordor from Windows - I'm using a separate host-only network (VMnet1) for that. To make this work on Vista, create a host-only network (in addition to bridging the ethernet card), assign it a static IP address (I'm using 192.168.3.0/24 for my "real" network and 192.168.8.0/24 for the host-only one) and set the default gateway to that address (address 192.168.8.1 / netmask 255.255.255.0 / gateway 192.168.8.1) - this'll fix the "Unidentified Network" problem in Vista.

Copying symbol files from Windows

Today, I added a cool new feature to the debugger: you can now add "directory maps" to the configuration file.

This is very helpful if you copy a symbol file from Windows to Linux. Simply add something like this to your ~/.config/MonoDebugger/MonoDebugger.xml:

<?xml version="1.0"?>
<DebuggerConfiguration fileversion="1.0">
  <DirectoryMap>
    <Map from="c:/cygwin/home/ichotolot" to="/home/martin/work" />
  </DirectoryMap>
</DebuggerConfiguration>

The debugger now automatically detects whether a symbol file has been created on Windows and converts the file names if necessary. This also works when using pdb2mdb to convert a .pdb to .mdb.

You could, for instance, mount c:\cygwin\home\ichotolot on /work/mordor using samba and use the configuration file to tell the debugger where to look for the source files.

Back from Madrid

After a really exciting week in Madrid, I'm now back at home and working on the debugger again.

We had a few really productive discussions about the debugger and sometimes, I got the impression that people didn't know that we actually have a working debugger before I did my little presentation at the meeting. Well, the debugger from SVN HEAD is considered stable and working - there will be an official release really soon.

Here are some pictures from our meeting:
http://www.flickr.com/photos/baulig/collections/

Ikea

Today, I went to Ikea in Koblenz to buy an armchair. It's really amazing to see how one room can completely change its shape within one hour:

Before driving to Ikea:

And after, with my new armchair:

Debugger currently broken

Unfortunately, I have some really bad news for all debugger users:
The debugger is currently broken and unusable because of recent changes in the Mono runtime. I can't give any ETA yet on how long it'll take to get it fixed since I need to do some research first. At the moment, I don't even know how to fix it, so I need to think about a solution first. Hopefully, I'll have more news after one or two days of research.

There's a Mono revision mentionend in the debugger's ChangeLog which is the last one which is guaranteed to work.

Debugger type system

Yesterday, I finally finished the work I started last week on the new debugger's type system.

For each local variable and method parameter, the debugging info generated by the runtime now also contains a MonoType *. On the debugger's side, we read this MonoType * and create the Cecil type from it. There have been some extensive API changes in the debugger's type systems since we now need to do a target access to resolve a type - that's something which would have been neccessary for generics anyways.

As a side-effect, we can now also unload and free the debugging information about types once an assembly gets unloaded.

Next week, I'll start to work on "real" generics support.

Debugger and Generics

About two weeks ago, I started to add generics support to the debugger.

After doing some research, I soon realized that things are a bit more complicated than I expected, and I need to modify some things in the debugger.

Initializing the debugging code

With yesterday's changes, the unmanaged debugging API has changed. If you're embedding Mono and want to initialize the debugging API, you only need to call mono_debug_init () - the old mono_debug_init_1() etc. hacks are now all gone (caution: do not call mono_debug_init_corlib()).

mono_debug_init() needs to be called before the first appdomain is created - after that, everything will go automatically.

Domain unloading and identifying threads

Today, I committed two important bug fixes in the debugger which both required a lot of code changes.

The most important user-visible change is that the debugging info is now stored on a per-appdomain basis - which means that it can be freed when the domain gets unloaded. We can now also unload symbol files. This required quite some changes - both in the runtime and in the debugger.

This wasn't only important to save memory, but also for correctness and robustness:
When a domain gets unloaded, all methods which were JITed in it are also freed - which means that the corresponding debugging info also isn't valid anymore. We now tell the debugger when an appdomain gets unloaded, so it can also free the corresponding symbol tables.

The second important change was a fix for a severe bug when attaching to a managed application. It is a bit hard to explain what this code exactly does, so this'll get a little bit technically.

On Linux, there is some kind of an API problem with the kernel and libc wrt. threads:

When you run a multi-threaded application, all threads share the same PID - so if you call getpid (), you'll get the same result in each thread. Internally each thread is a seperate `task' for the linux kernel and it also does have it's own pid - which is also called LWP - but there's no (portable) way of getting that from user-level code. That's why user-level code normally uses pthread_t (which is returned by pthread_self()) to identify a thread - we call that a TID.

When attaching to a managed application, the debugger needs to get some information about all the managed threads - precisely the LMF address, which is required to generate managed backtraces when we're stopped inside native code, and some other internal information. The only thing the runtime can provide is a TID to LMF mapping - but the debugger only knows about the LWP and not about the TID.

So I had to find a way of getting the TID from an LWP - the old code simply called pthread_self() in each thread, but of course this approach has several problems. The most severe one is that this obviously doesn't work for core files. It also doesn't scale very much to a large number of threads since invoking methods in the target is extremely expensive. Another important point is that I want to be able to generate a backtrace before executing any code in the target - that's very important if the target is not responding anymore.

After doing a lot of research, I finally decided to use glibc's thread_db library - this can be used to get information about all the threads, so after some work I finally had it working.

We still need to use some really bad hacks for core files, but they're now working as well.

So to summarize, almost two weeks of hard work - but now, attaching and domain unloading is finally working :-)

Tomorrow, I can finally move on to generics ...

Domain unloading and the debugging code

Yesterday, I got an important and interesting bug report:

We currently do not support domain unloading in the new debugging code. This means that when a domain is unloaded, we do not free the debugging info. To make things worse, some internal hashtables also contain references to the MonoMethod *'s which will lead to crashes later on.

Freeing the debugging info is not so easy since the debugger may access it at any time, so we need to tell the debugger first. However, I got a really good idea this morning and am currently working on it.

Native line numbers

After finally fixing the performance leak last week, I started to work on another rather difficult problem.

It all started with a rather trivial bug report:
Miguel sent me a small moonlight application and said that the debugger would crash when trying to get a backtrace from within an event handler.

Sounded like a rather trivial problem, so I started debugging ....

Things turned out to be a lot more complicated - after spending a lot of time doing some research on inline methods and extensive reading of the dwarf spec, I found a huge design problem in the code which reads the line numbers from the dwarf debugging info.

In DWARF, line numbers are stored in a separete section of the symbol file and not in any way related to methods. The debugging information entry for a method (DW_TAG_subprogram) only contains a start and end address - but no information about where the line numbers for that method are stored. The line number table, on the other hand, only contains a mapping from addresses to line numbers and contains no information about methods. To make things worse, the line number table does not need to be continuous and there is no separation whatsoever between different methods.

This makes it rather complicated for the debugger to read line numbers. It looks like the old code only worked by accident since older versions of gcc always created a continuous line number table - g++ doesn't (I could even create a test case where g++ produces a non-continuous line number table where the test case doesn't contain any c++ features at all).

To make things work, I had to do some changes in the debugger: rather than having on LineNumberTable per Method, each Method now contains a reference to a LineNumberTable but it doesn't "own" that table (ie. it could be shared between different methods). I had to move a lot of code around for this; for instance, the LineNumberTable now longer contains the method's start and end row etc.

While working on this, I soon began to realized that a change like this would have been required anyways when supporting compiler generated code. From the debugger's point of view, the difference between an anonymous method (in C#) and an inline method (in native code) is not so big at all.

If everything works out fine, I should be able to finish the new dwarf code till the end of the week and then start to work on compiler-generated code on Monday.

Debugger Performance

Last night, I finally found the performance problem in the new debugger code - and it was really trivial:
We were missing a mono_debugger_unlock(), so we got a deadlock on exit. There's a 2 second timeout in mono_domain_finalize() when it's called from mini_cleanup() and we have 39 nunit tests - makes exactly 78 seconds in total.

Things are now fixed and the new debugger code is as fast as the old one :-)

More performance problems - I feel like I'm seeing ghosts ....

On Saturday, I finally completed the new symbol table code.

Last week, I already noticed a really huge performance leak, so I spent some more time investigating. Unfortunately, I'm getting more and more confused the more I play with this.

At the moment, I feel like I'm seeing ghosts .....

Running the complete debugger test suite in the old debugger takes about 65-70 seconds. In the new debugger code, it takes about 150-160 seconds. So that's more than twice the time and we must have a really huge performance leak somewhere.

So I started investigating. To make sure it's not the new symbol table code in the runtime, I wrote a small script which compiles Mono.C5 100 times and uses mono --debug each time when invoking gmcs.

I was really surprised when I looked at the results:
It took 24 minutes 57.195 seconds with the old code and 24 minutes 34.012 seconds with the new one - which means we're now 23.183 seconds (or 1.55%) faster than before !

This means the real problem must be somewhere inside the debugger ....

More symbol table stuff

My new breakpoint code is coming along really good :-)

Yesterday, I got all the required features working so I started testing. The good news is that this code actually works - multiple appdomains and generic instances are working just fine :-)

The longer I'm working on this, the more problems I find in the old symbol table code. There is, for instance, a piece of code which never worked for the past 2 years - but nobody ever noticed because it can only be triggered if you run a really, really huge application with ---debug. I only figured it out by accident while I decreased the limits to do some performance testing. There are also race conditions when we need to start a new symbol table since the debugger might currently be reading it.

However, this shouldn't be too bad - I already have some really good ideas which'll not only increase stability but also performance.

Method lookups, symbol files and breakpoints

Last week, I started to work on true multi-appdomain support for the debugger. Soon, I realized that there's also another problem: generics.

Both appdomains and generics have one thing in common: a single method may be JITed multiple times. Because of that, I decided to use a common interface in the symbol table code for both. Now, each method in the source code may have multiple addresses.

To make things easier, I decided that in the case of generic methods, we may not insert a breakpoint on one particular instantiation - when inserting a breakpoint on a generic method (or any method in an instantiated generic class), it always affects all instantiations of that method. This doesn't only make the breakpoint code a lot easier - the fact that a method is JITed multiple times for multiple instantiations is also more or less an implementation detail of our current JIT.

When inserting a breakpoint, we first need to check which instances of that method have already been compiled and then physically insert a breakpoint on each of them. Each time, a new instance of the method is JITed, the debugger gets a notification, so it can also insert a breakpoint there.

One important thing when implementing the new code was that it should also scale well to a large number of appdomains and/or generic instantiations.

A first idea from me was to get a notification each time a new appdomain is created and then insert a breakpoint on each of them. However, since callbacks from the debugger to the JIT are expensive and reading a large chunk of data from it is rather cheap, I realized that this doesn't scale very well. Because of that, the JIT now gives the debugger a list of method addresses.

I've already got some preliminary code working, just need to do a few more tests. Before I finish this, I'd like to do some tests with a large number of appdomains and generic instances. Hopefully, I'll be able to finish this tomorrow or on Wednesday.

It can happen to the best of us .... and a long day with a lot of annoying paperwork is finally over

Until yesterday afternoon, I thought things like EC card fraud can't happen to you if you're always careful with using your card - well, I was very wrong about that, it can happen to anyone :-(

Yesterday, after lunch I routinely checked the balance of my bank account using phone banking and was really shocked to hear that it was -4785 EUR - at the beginning of a month, something must be wrong. I immediately asked for customer support to check what was going on. First thing the customer service agent said to me was something like "Oh shit, f... we have a problem :-(" - someone was using my card to withdraw money from ATM machines in Bankog - there were a bunch of such withdrawals of 10.000 whatever-their-currency-is (which converts to about 220 EUR).

So today, I basically spent the whole day doing paperwork. First I had to go to my bank's local branch to get a detailed statement of all these withdrawals - then go to the police to file a report and then fax a copy of it to the bank.

The good news is, the bank already told me that I'll get my money back - since I still have the card, someone must have copied it when I was using an ATM or paying with it at one of these pay-at-pump gas stations.

It was just very frustrating to do all the paperwork and spend hours of waiting. The local police was extremely busy today and the officer I was supposed to talk to was called to several different operations while I was there, so I had to wait a lot.

Sometimes it's a little bit scary to wait that long at a police stations. They send you into a small room to wait till an officer has time for you and if you wait there for some time, a lot of other people come and go - most of them have problems like "Oh, I got my apartment searched yesterday, and you seized XYZ, where can I get it back?" or "Oh, I'm here to surrender my driver's license" or "Oh, I damaged XYZ while I was drunk and was asked to report here".

Live Free or Die Hard

Yesterday, I watched Bruce Willis's new movie Live Free or Die Hard - which was really excellent :-)

This was one of the movies I really wanted to watch in the original version and also without any subtitles. The premier in Luxemburg is on July 4th and they also have these annoying subtitles, so I decided to drive to Düsseldorf and watch it there - a 300km drive from where I live, but it was really worth it.

One year and one day ...

... have passed since I blogged last time.

That's really a very long time and a lot of things have happened in the meantime. Since about November, I'm back hacking on the debugger full-time and there some exciting news about it.

On May 9th, I released version 0.50 "Dublin" of the Mono Debugger. This is the latest stable version and also a milestone. It's one of the best releases ever and I'm really happy about it.

After that, I started some really large code cleanups and rewrites on a separate debugger-dublin branch. I had several goals with this project:

• Support multiple appdomains
• Rework the breakpoint code so we don't need to stop in Main() and we can also support static .cctors
• Don't change the application's flow of execution as much as we're doing at the moment.

I'm now about one week away from implementing full multi-appdomain support, so I think I'll wait until that is done before merging my code back. I've posted a very detailed summary of my latest changes to our internal mailing list, which is also available in SVN.

Read the full story ...

Released version 0.30 "Koblenz" of the Mono Debugger

It has been some time since the last debugger release, but now it's finally ready - version 0.30 "Koblenz" of the Mono Debugger has just been released :-)

The big news is that there are now binary packages available for a number of distros - they can be found at http://www.mono-project.com/Downloads. The source tarball can be found here.

There are a lot of stability improvements in this release and the debugger has also been tested on i386.

Btw. the release name is some kind of an "in-joke" - I'm reserving the name "Cologne" for a stable, 1.0-like, release - and Koblenz is about half the distance to Cologne from where I live.

Update: I have been asked about the binary packages since they are a bit hard to find. On http://www.mono-project.com/Downloads, select your distro - for instance "SuSE Linux Enterprise 9 x86_64" - and then follow the "latest" link. This'll lead you to http://www.go-mono.com/download-latest/sles-9-x86_64 where you find the debugger package in the "Development Tools" section.

Switching on Nullables

Yesterday, I implemented a cool new feature in GMCS: switching on nullables - you can now write code like this:

And for all you fans of the World Cup 2006 - I am very happy today:

I got two tickets for the Paraguay - Trididad & Tobago game in the Fritz Walter Stadium in Kaiserslautern (I know, the website is in German, click on Bildergalerie to see pictures). That stadium - also called "Betzenberg" - is the home stadium of my favorite team, the 1. FCK (unfortunately, they're not playing in first league anymore next season) - and I already watched a lot of games there in the past ....

Baile Átha Cliath

I'm back from a wonderfull weekend in beautiful Dublin ("Baile Átha Cliath"), met a lot of nice people and had a lot of fun there :-)

Friday and Saturday, I spent most time going from one Pub to the next - mostly around Temple Bar - and tasting a lot of Irish Beer ...
The weather was absolutely awesome and so on Sunday, I did some sightseeing, walked around in the park and then finally found myself in another nice pub drinking even more Guineess and other Irish beers. I had to wake up really early on Monday morning, my plane was supposed to leave at 7:00 am - but somehow I still managed to miss it - it wasn't even that late, just about 6:40 am when I arrived at the airport, but they already closed checkin and I wasn't allowed to board anymore :-(

However, Air Lingus is really a very cool airline - they allowed me to change my ticket by just paying the fare difference plus a small fee. Unfortunately, the second flight on that day was already completely booked out as well as the 7:00 am one on the next morning.

So at about 6:55 am on Monday, when I realized that I was stuck in Ireland for another 34 hours, I decided to see more of this beautiful country and rented a car. It turned out that driving on the wrong side of the road wasn't even that complicated at all and the Irish people are really very nice and helpful - thanks a lot again for helping me out when I got stuck with a flat tire somewhere in the middle of nowhere, with nothing but cows and sheep around !

Galway and Cork were both really nice and Galway has some beautiful beaches. Unfortunately, it was very cold and rainy when I left Germany, so I didn't even think about swimming and didn't pack anything - so I just walked around the beach a little bit. I finally arrived at Cork late at night and was very happy to find a pub which was still open and they were also playing live music there :-)

The next day, I got up very early and started driving back to Dublin - I almost made it to the airport, just another 15 kilometers and plenty of time left - when I got stuck in a really bad traffic jam. For the first two hours, nothing was moving at all - I could have walked to the airport in that time, but I had to return that rental car ... - and then it started moving very slowly. I lost over 3 hours in that jam and when I finally arrived at the airport at 4:55 pm, I didn't even bother going to the checkin counter (my flight was supposed to leave at 5:00 pm) but run upstairs to the airport bar and just made it in time.

When I arrived, I saw my airplane already being pulled away from the gate and slowly moving towards the runway .... so I quickly ordered and when it took off, I waved goodbye with a Guinness in my hand ;-)

As I already said, Air Lingus is a really cool airline - changing my ticket a second time wasn't any problem at all and I got the 7:00 am flight the next morning.

Off to Dublin ....

Tomorrow morning, I'm heading off to Dublin, the beautiful capital of Ireland for a long weekend of sightseeing, Guinness-drinking, partying and having fun :-)
I'm really looking forward to it - it has been a long time since I've been in Dublin last time ....

Just the right time to relax before a busy and stressfull week - next Thursday, I'm getting new windows in my apartment. In general, that's a very good idea since my old windows are probably older than I am and I'm really happy that the landlord wants to replace them. However, I'm a bit scared of the dirt and dust this operation will make. At least, I'll be some work moving my desk away from the window and covering all furniture with covering sheets. And then, on Friday, it's the big spring cleaning day for me ...

Ok, time to leave to the Irish Pub to get another pint of Kilkenney before leaving :-)

Released version 0.20 "Bruxelles" of the Mono Debugger

After almost two months, I finally released a new stable version of the Mono Debugger - it's called 0.20 "Bruxelles" and can be found at http://primates.ximian.com/~martin/debugger/mono-debugger-0.20.tar.gz.

There are lots of improvements in this release:

• This version of the debugger requires the latest Mono from SVN; it won't work with the Mono 1.1.15 release.
• We now preserve breakpoints across different invocations of the same target.
• Big API cleanups.
• Cleaned up method lookups; there's a very complex test for them in test/testsuite/TestMethodLookup.cs.
• Lots of stability improvements.
• Fixed a very old GC bug which was preventing us from running xsp inside the debugger.
• Fixed a very old race condition which was causing crashes at exit.
• Fixed line numbers in stack traces.
• Added experimental support for hardware watchpoints - I needed them to track down a race condition.
• Unfortunately, a `make check' now requires even more memory than before; I'm still investigating the issue.
  I'm currently using `ulimit -Sm 1000000 -Sv 1000000'.

Since this release contains so many stability improvements, I decided to move to a new versioning scheme - this release really deserves being called 0.20 rather than 0.14 ;-)

The next releases will be "Dublin", "Sidney" and "Cologne".

Released version 0.12 "Portland" of the Mono Debugger

Version 0.12 "Portland" of the Mono Debugger has just been released - it can be found here :-)

• This version of the Mono Debugger requires Mono 1.1.13.4.

  For those of you using SVN, you need the latest Mono from either SVN HEAD or anything from the 1-1-13 branch which is at least Mono 1.1.13.4.

• The debugger no longer uses a mono-debugger-mini-wrapper, it's now using the normal mono instead.

• We can now attach to processes and examine core files - this still needs some testing, so bug reports are very welcome ...

• We no longer run the backend in another process/appdomain; most of the remoting stuff is gone to make the debugger faster and more reliable.

• This release is named after Portland, Oregon.

I'm still very busy fixing GMCS bugs this whole week, but next week I'll be back in debugger land - so if you have any problems or bugs, please lemme know.

C5 1.0 hits Mono SVN

As Miguel already mentionend in his blog on February 1st:

Peter Sestoft released version 1.0 of his C5 generics library for C#, it is available here:

C5 provides functionality and data structures not provided by the standard .Net System.Collections.Generic namespace, such as tree data structures, heap based priority queues and hash indexed array lists and linked lists, making it more comprehensive than collection class libraries on similar platforms, such as Java. Unlike many other collection class libraries, C5 is designed with a strict policy of supporting "code to interface not implementation".

C5 is documented on his technical report and he also has the docs online.

Importing this version into Mono SVN was a real challenge since it exposed many bugs in GMCS and the runtime. Now, over a month later, I finally had the time to fix all these bugs and I'm really happy that we now have a working C5 in Mono SVN again :-)

Today, version 1.0 of C5 has finally hit Mono SVN in HEAD and the 1.1.13 branch.

The end of the `mono-debugger-mini-wrapper'

As of today, the Mono Debugger is now using the normal `mono' to execute managed code rather than using its own `mono-debugger-mini-wrapper'.

This means that you need a recent SVN version of Mono from either HEAD or the 1-1-13 branch: everything after r55889 is ok.

There are two big advantages of the new setup:

Packaging

Since we no longer need to worry about finding the "correct" wrapper, packaging and relocating the debugger is now much easier.

Testing a different Mono

You can now use a different Mono to debug your applications.
This can be very helpful if you're trying to find a runtime or JIT bug: you can use a stable version of Mono to run the debugger and debug an application using a newly compiled, unstable one.

The required mini changes are both in HEAD and in the 1-1-13 branch, so these two versions are explicitly compatible with each other - you can, for instance, use the 1.1.13 Mono to run the debugger and use Mono from HEAD in the target application.

Back in cold Germany ....

After two wonderful weeks of vacations in beautiful Guam, I'm finally back in cold Germany. Before I start looking at all these GMCS bugs in bugzilla, here are a few last pictures:

Paradise is Chamorro

5 Things to Know about Guam

• Where the hell is Guam ?
  Guam is located at 13° 26′ 31″ N 144° 46′ 35″ E
• How does Guam look like ?
  Well, depends on how do you define paradise ....
• Where do I get a map ?
  It took me some time to figure this out when I came here. The best way of getting a map is either asking in one of the big hotels or at Two Lover's Point
• Which language do they speak in Guam ?
  Guam is an US territory, so the official language is english.
• How are the native people of the island called ?
  They're called Chamorros

5 Places you Must Go while you're in Guam

• Tarague Beach
  If you like a nice and quiet place to chill out and enjoy paradise ...
• The Gapgab
  Almost the westernmost point of the USA and a nice place to go snorkeling
• Ypao Beach
  Located right next to the big hotels. Unfortunately very crowded by tourists.
• Ritidian Point
  Another very nice beach which is located in the wildlife preserve.
• Two Lovers Point
  Very nice overview over the island and also a perfect place for your next wedding ....

5 Pubs and Bars

• The Tower of London
  A very cool english pub with all sorts of foreign beers.
• Jimmy's Beach Bar
  Having fun on the beach ...
• Hard Rock Cafe
  Of course, there's a Hard Rock Cafe in Guam.
• Brewery
  This isn't just a brewery, but a cool nightclub where you can dance and have a lot of fun.
• Denial
  If you want it more extreme ....

Some more pictures

A warm and sunny place ....

Today was my last day of work before my christmas vacations. Now it's time for me to start packing my bags and get ready to leave - on Monday, I'm flying to Guam for 16 wonderful days :-)

At the moment, it's 81 F or 27 C there and the beach is waiting for me ....

Before I'm leaving, I also have a little Christmas present for you guys: after 18 months, there's finally another debugger release - it's called 0.11 "Guam" :-)

You can get the debugger at http://primates.ximian.com/~martin/debugger/mono-debugger-0.11.tar.gz. This release requires Mono 1.1.11, and - unlike most previous debugger releases - this one works with the released Mono and doesn't require you to use SVN. The debugger tarball also contains a pre-built version of Cecil, so you don't need to get that separately.

The documentation of the debugger can be found on the Mono website at http://www.mono-project.com/Guide:Debugger

Ok, now it's time for me wish you all Happy Christmas and a Happy New Year - I'll be back on January 9th - if I can get any internet access in Guam, I'll try to post some pictures ....

setXid() madness

Yesterday, I stumbled upon a very weird thing in glibc when tying to figure out why applications using gnome-vfs were hanging in the debugger. In multi-threaded applications, all functions in the setXid familty (ie. setuid(), seteuid(), setgid(), setegid()) are not implemented using a system call, but instead send a signal to the process and have the signal handler do the system call in each thread.

The implementation of all these functions first checks entry 51 in the __libc_pthread_functions vtable: if it's NULL, a normal system call is made. Otherwise, the call is forwarded to that function - it's called __nptl_sigxid() and what it does is sending a signal to each of the process'es threads - so the signal handler is invoked in each thread. After that, it waits until all the signal handlers have finished and returns.

This is very bad when running in a debugger since one or more threads may be stopped - in which case it'd block forever. Sometimes the debugger also needs to do some "magic" which is normally invisible to the user, but causes some threads being stopped (when stepping over a breakpoint on the current instruction and when dealing with the garbage collector, for instance).

Unfortunately, gnome-vfs calls seteuid() a lot of times even if it's runinng as a normal user and not as root. It's explicitly forbidden to debug any setuid/setgid applications with mdb (the linux kernel doesn't allow this), so the only case where setuid() actually does something is when running as root.

In order to make gnome-vfs work reliably in mdb (debugging applications which use gnome-vfs is what a lot of people want), I added a really bad hack to the debugger: it sets a breakpoint on __nptl_setxid() and whenever it's called, it makes it return 0.

Of cource, this has the side-effect that if you're running as root, the application won't be able to drop privileges anymore and it'll always run as root. You can still run mdb as root to debug most applications, but you may run into problems if you have some deamon-like application which absolutely needs to change its UID for some reason.

However, the positive effect is that any application which is using gnome-vfs is now working reliably in mdb.

I really hope that some day the kernel people will implement the setXid() functionality in the kernel rather than having this weird signal-based approach in user-space.