The Mysterious Breakpoint

(Or how even Microsoft programmers sometimes forget to clean-up..)

While working on a piece of software that relied on a ActiveX control, I regularly found my debugger stopping at an address somewhere in memory that was not part of the program. First I tried to find out where the bug was, then I found that just continuing the program worked fine, and that there were apparently no side-effects. But it kept stopping there, no matter how much of the debugger features I disabled. So I decided to get into it a bit deeper and found out it always stopped at an address named 'DbgBreakPoint' in module NTDLL.DLL. It looked like it stopped on a breakpoint, but not one that I had set. And the 'Disable all breakpoints' option in the Borland C++ Builder debugger did not make any difference. It really looked like Microsoft forgot to remove a breakpoint from one of the most important DLL's in Windows. A bit of Googling made clear that this was indeed the case, and also that it was not easy to get around it. Patching NTDLL.DLL appears impossible since it is one of the most 'protected' core-pieces of Windows. If you try to modify it, Windows will just 'restore' it on start-up. And if you are clever and modify the backup copy it uses for this restoration, Windows just won't start at all...
So the only way around seems to 'patch' the DLL in memory after it is loaded by your program by overwriting the memory location containing the breakpoint with a 'NOP' instruction. Here is the function to do it:

void PatchINT3()
{
unsigned char NOP=0x90;
void* pNTDLL;
DWORD BytesWritten ;
FARPROC Address;
if (Win32Platform != VER_PLATFORM_WIN32_NT){ return; }
pNTDLL = GetModuleHandle("NTDLL.DLL");
if (pNTDLL == 0) {return;}
Address = GetProcAddress(pNTDLL, (LPCSTR)"DbgBreakPoint");
if (Address == NULL){ return;}
unsigned char Byte= *(char *)Address;
if (Byte != 0xCC) {return;}
try
{
if (WriteProcessMemory(GetCurrentProcess(), Address, &NOP, 1, &BytesWritten))
{
if(BytesWritten == 1)
{
FlushInstructionCache(GetCurrentProcess(), Address, 1);
}
}
}catch(EAccessViolation &E){}
//Do not panic if you see an EAccessViolation here, it is perfectly harmless!
}

I suppose anyone who got to here has enough knowledge of Windows programming to see how it works :-)
And so, by calling PatchINT3() at the start of the program the breakpoint is erased and won't show up again.
(Thanks to Peter Morris at HowToDoThings , who wrote the Delphi version that I translated here)

Serial Communication Trouble

Ever since the 'real' serial comport (the hardwired connector on the rear of your PC) has been replaced with all types of 'virtual' alternatives like IR ports or Serial to USB devices, it has been a cause for all kind of trouble. And I've seen many of them, but the one I found this morning was new to me. Suddenly one of the programs I've written myself showed 18 comports to choose from. Now the program uses a dedicated function to enumerate the available ports, so obviously I thought it had something to do with a bug in my software. Also because the 'Hardware manager' of the PC only showed the two comports that are actually available. On the other hand I found that some other software that uses serial ports showed the same behaviour.
So I looked at the registry ( HKEY_LOCAL_MACHINE\HARDWARE\DEVICEMAP\SERIALCOMM) to find out which port were actually registered. And this showed all 18 as \Device\VComxx, where xx is the port number. So obviously something was creating virtual serial com ports. But what? I did not add any hardware to my PC that could cause this, nor did I install any serial com port related software or drivers.
So finally I located in the System Properties -> Device manager under 'System Devices' two suspicious entries: 'NCF Virtual serial bus enumerator' and 'VXCOM port'. The first turned out to be the 'Nokia Communication Framework', something I installed long ago in an attempt to communicate with my Nokia phone. After searching on the web for the second one, I found only a few entries, but it seems this one is related to the Windows Mobile 6 SDK, which I indeed installed last week. And when I disabled this driver, the problem was gone.
Apparently VXCOM is a virtual com driver used for the SmartPhone emulator. Why it reserves 18 ports at startup, even though I never used this emulator is a mystery to me...

Mouse toTrackball to Centipede

Born in '65, it's inevitable that I grew up playing arcade games in the local arcade, re-discovered the joy of the simple games of those days and found my way to the MAME project.
I downloaded the the MAME executable and a bunch of (illegal...?) games, and I carried it with me several years, copying it to every new PC, and occasionally playing some of my favourite games. What always created a slight itch was the fact that playing on a computer keyboard does not really come near the 'real arcade experience'. So, like many others.. I would love to build a real arcade cabinet, with real joystick, trackball, spinner and fire buttons. Of course I lack the time to build it, a place to store it and even time to play it, but the idea remains. Looking around I found several on-line shops that sell all necessary parts from controls to marquees to make it look exactly like the original, but adding it all together it gets quite expensive. Certainly controls like a trackball and a spinner are costly items. And actually a spinner does not seem like a difficult thing to make yourself, but I decided to start with the trackball, since I already owned this. It is a Wico-trackball that was manufactured in 1983(!) for the then popular TI-99 home computer. Actually this seems bad timing, since TI exited the home computer market by the end of '83... Well, I bought the trackball a few years later for a few bucks, and first interfaced it to my Commodore 64, but hardly ever used it. Since a trackball and a standard computer mouse (the ball type) are very similar I thought it would not be too difficult to combine the two and have a 'mouse compatible' trackball. Fortunately 'standard' mouses can be found in large piles lying around unused in almost every office, so I soon found a suitable Logitech USB type that no one cared about anymore. First I thought it should be possible to use the original optical switches of the Wico, and just wire these to the mouse electronics. But since neither the mouse nor the trackball switches have any type numbers stamped on them so I could not easily compare the connections and make a new routing plan, I decided it would be easier just to remove the switches from the mouse, and mount them inside the trackball, completely discarding the original trackball electronics. So I just removed the Infrared LEDs and the opto switches from the mouse board, mounted them on a piece of prototyping board, and re-wired them to the original contacts. The image here shows the result. Bottom right you see the original mouse electronics. Bottom left you see the piece of prototype board holding the LED and the opto-switch. Right top you see a little piece of the second board just before it's put in place. Top left you also see the contacts of the 'fire' button. The wires from this I just cut to the right length and soldered them to the rear of the switch that used to be the left mouse button.
And it works great!. Finally I can play one of my favourites (Centipede) the way it was intended. So now I'm ready to take the next step: build a 'Rotary controller' or 'Spinner' for my all time favourite game : Tempest .Right now I'm chasing another unused mouse...

The Big Step: Microsoft Visual Studio

So it finally had to happen. I purchased and started to use Visual Studio 2005. After using Borland C++ Builder for more than 10 years for all my Windows (and even DOS) programming, the Pocket PC platform turned out be the decision maker. From the start the only way to write programs for the Pocket PC platform has been through Microsoft. And they also made it very easy by giving away the free Embedded Visual Basic, followed by Embedded VisualC. But today, now everybody has a Pocket PC, none of these tools is being supported, and they also refuse to work with the new Windows Mobile 5 devices. Now this could be a great starting point for some Microsoft bashing, but actually I read some explanation on a MS-Developer blog that makes sense. Since the embedded versions where completely separate projects within Microsoft, the development and support started to suffer from the fact that the developers also worked on the commercial Visual Studio product. And it's just a fact of life that supporting a commercial product has a higher priority...
But I've written all my software using EVC 3 and 4. And so I struggled some time by using old PocketPC's for debugging, actually waiting for Borland to come up with the 'Delphi for PocketPC'. But I don't think this will happen for a while, and I even wonder if they ever will produce something for the PocketPC platform. So I finally had to get Visual Studio 2005 to continue supporting my embedded Visual C apps. Fortunately I could go for an Visual Studio 2005 Upgrade, because I already owned C++ Builder, making it a bearable € 250,-.
And now I must agree, the Visual Studio environment a a giant leap forward compared to the old EVC programs. The editor is much more up to date, it compiles faster, has a better debugger, and (surprise, surprise) also supports direct emulation on your desktop PC, even when your program uses ActiveX controls or external DLL's !.

Creating Q-Web

To visualise data that is related to geographical locations (e.g. the local weather) nothing beats a good map with some clear symbols that reveal data from a database. The following series describes my route towards a database driven ASP.NET application that shows data as collected by the 'Discharge measurement' stations. The company I work for (Qmetrix) supplies these stations to local water management authorities. They measure the flow of water in a river or stream, combine it with the water-level and the local depth-profile and calculate the total cubic meters of water per second that passes by.
First I needed a nice map. I used MapBuilder, the excellent tool to manipulate Google maps. Now what I need for the webpage is a image for the area of interest, for which I know the exact coordinates of the left-top and right-bottom corner. Here is how:
In MapBuilder first click 'New Map' and enter a (short) name and a description. Then return to 'Edit Map'. Disable the 'Automatic Zoom' feature. Make sure the area of interest is completely visible. (Find it quick by just typing the name and country of a city that that in the area in the "Location Search & Quick Navigation" box )
Click in the left top corner, this will show a blinking marker. Select a small marker type, type 'Left Top' in the 'Caption' box and click 'Add'.
Repeat these steps for the 'Right-Bottom' marker. Now click 'Save Center, Zoom, Maptype'. Select 'Map Controls' in the menu, and disable all visible features. Don't forget to click the 'Save' button at the bottom of this screen.
Now choose 'Preview' in the menu. It might be required to shift the map a little to get both markers in the right place.

If required you can go back to the 'Map Controls' screen, and increase the map width and height. Now if you don't want to include the Google and Mapbuilder trademarks you must make sure they fall outside the area as enclosed by the two markers. If the map is right, capture the whole screen, using a screen-capture utility or just [Shift][PrtScreen]. Load the captured image in an image editor, and cut out the exact area Top left to Bottom right. Save this as a new (.jpg or .png) image.
Now go back to MapBuilder, and make a note of the exact coordinates of Top-Left and Right-Bottom. (I usually just click on the markers, and Copy-Paste the 'Latitude' and 'Longitude' fields to a Notepad text file.)
While this text file is open you might also write down the exact sizes (in pixels) of the map image we just created. Usually this can be found using the image editor, but you can also locate the file in Explorer, right click it and select 'Properties'. Hidden under an 'Advanced->' button you find the exact size of the image in pixels.