Saturday, 10. October 2009
DEP/NX Breaks Older C++ ActiveX Controls in .Net 3.5 Apps (including Web Apps)10/10/2009
Thought I'd write this up in case it saves someone else a few days of debugging in Windows 2008 Server. The cryptic error message you'll get in your Windows Forms or ASP.Net apps is "Unable to get the window handle for the 'AxBlahBlah' control. Windowless ActiveX controls are not supported." If you look at the stack trace a bit more, you'll find an Access Violation (thanks to Tom Frey for that hint). The reason for this is that some older C++ ActiveX controls that linked to MS' very own ATL libraries are not NX/DEP safe (i.e., they run code inside data segments!); ATL versions 7.1 and earlier (basically anything before Visual Studio 2005) do this funky stunt. So, your choice is to run your app w/ DEP disabled or if you're lucky, you can recompile those ancient ActiveX controls which is another PITA in itself because you have to get it to compile in a later version of Visual Studio and these controls inevitably have C++ quirks.
Windows 2008 is also missing linkinfo.dll by default, so if your ActiveX control uses wininet.dll, it can't be registered until you install the Desktop Experience feature (which includes Windows Media Player, etc.) in Win2008. Why oh why? :-P At least you can uninstall that feature afterwards; the linkinfo.dll seems to stay around.
Ken's Blog
-