I run PinballX, FX2 / 3 and DOFLinx (obviously ) as standard on my cabinet, so I am sure your setup will work. I have the backglass disabled and the DMD repositioned to a third screen.
There are two parts to think about here;
1) The loading of the B2S correctly
2) FX2 Starting up OK
Is this a matter of focus? ie if you Alt-Tab do you find the running game? If so there are options for this as well (FX2_FOCUS=)
Here are a couple of suggested tests to help diagnose where the issue may be;
1) Set DEBUG=1 in DOFLinx.INI and use the right-click Test B2S option to try loading and running an effect within a B2S backglass. (Note this test routine does use the FX3 B2S path for its back glasses - same test though)
2) If you start FX2 from a command line with the arguments to start a game immediately what do you get?
3) Set DEBUG=2 and run the entire sequence, this will log all the activities from DOFLinx through the sequence - I doubt there will be anything telling, but it will help more than the very short DEBUG=0 log you have so far.