@Tom Speirs I created a program that will give the proper FFMpeg commands. Full source code is here: https://github.com/xantari/PinCabScreenConfigurator
There are a some other features in it right now (see documentation at the above URL), but I wanted to give you this so you could implement the fix for the FFMpeg command generation in your code base.
The main command you want to look at is the FFMPeg toolstrip command here: https://github.com/xantari/PinCabScreenConfigurator/blob/f51f0195e912c6696311fa12e5b2e98d01c40c88/PinCabScreenConfigurator/MainForm.cs#L136
You will want to take the DMD coordinates and screen configuration information from PinballX.ini and fill in the proper DisplayDetail object and it will do everything for you.
This would purely use what the user configured in the PinballX.ini file. No PinMame, DMDExt, or anything else would interfer with the details. You just specify what is your displays (Playfield, DMD, Backglass, Apron, Topper) and it does the rest.
A few other observations:
1. Only display the connected displays in the PinballX Settings program. Right now it displays display 1 through 5, even though I don't have a 4th and 5th display (see source code on how I populated the ListBox list for this)
2. The PinballX settings program just says "DIsplay 1", "Dispaly 2", etc. Look at my source code, you can show actual connected monitor EDID information and the windows display name. See how that is done in ListBox on MainForm
3. There is a screen validation function you could implement in your settings program to ensure no screens have negative offsets which would save a bunch of forum postings i'm sure, check the above source code on how that was done.