Managed to dig up some rudimentary information on PROGMAN. That plus some DDESpy traces allowed me to figure out how to respond to a DDE initiate request, and then accept commands. Seems like it's good enough to just blindly ack all the commands without doing any work, which is great. There appears to be no real return values.
It seems some installers expect PROGMAN to always be running, while others are able to launch it if it is not. I decided to detect the broadcast WM_DDE_INITIATE message and auto-launch my stub PROGMAN handler in response, so it's effectively there without needing to explicitly launch it.
Then I spent the rest of the day in scheduling hell as these new code paths revealed some problems in my task scheduling logic and how cross-task messages were being sent, along with interactions between thread and task scheduling. Think I might finally have it working ok now, but my brain is seriously fried!
