[APP] Recently Added/Update tables creator

[Mike_da_Spike]

25 minutes ago, scutters said:

About 150 enabled at the moment.... Must be about time for a cull to get it back down to < 100! 

[simbamame]

@Mike_da_Spike hi mate, i download and ran this, amazing, worked 1st time with no problems at all, it created the recently added tables xml correctly and in the right folder.

Many thanks.

Ian

[scutters]

@Mike_da_Spike

Hi Mike, just to let you know that the Recently Added & Recently Updated Tables xml functionality is working great for me. Not a beta anymore in my view!.

Thanks again for this, I now consider it one of my essential pinballx utilities 

[Mike_da_Spike]

4 hours ago, scutters said:

@Mike_da_Spike

Hi Mike, just to let you know that the Recently Added & Recently Updated Tables xml functionality is working great for me. Not a beta anymore in my view!.

Thanks again for this, I now consider it one of my essential pinballx utilities 

Thanks man. I appriciate it. Im not a developer, so wasn't sure if this could help someone.

I use this tool too. I run it every time I add or modify tables. Just run in 30 seconds and creates all. 

Glad I could help you with this and thanks for testing it and reporting issues. 

[simbamame]

i agree with @scutters release it as a finished plug in, ive encountered ZERO problems with this and have used a fair bit as i currently add about 10-12 tables a week from various sites.

[Mike_da_Spike]

Wow. Thanks guys. I appreciate the nice words. I noticed some bugs in the logging. I will fix that and call it final version.

Edit : Only did a text change in the logging (not a big deal) and put the version to 1.0
I'm also in progress to create a cleanup part for the Media.
This means for myself I cleanup :
 

• Remove all data in the Media folders that i specify :  "Backglass Images","DMD Images","DMD Videos","Real DMD Color Images","Real DMD Images","Real DMD Videos","Screen Grabs","Screen Grabs Backglass","Table Images","Table Images Desktop","Table Videos Desktop"(I use cabinet mode and want to use videos and no images )
• Remove all the *.f4v files in Table Videos and Backglass Videos (creating my own MP4 files with gamemanager)
• Remove all  videos in 'Table Videos' and 'Backglass Video's that doesn't have a corresponding name using by this tool (lot of videos has a version number in it. so no need to keep multiple versions of a video)
• RealDMDVideos (in VP9 and VPX) that are not 7 seconds long (I create my own videos with a small script of 7 seconds)
•  
• With all this above I run it for the first time and remove around 8.9 Gb of files I didn't use.

This is just for only my cabinet, but is there any interest to have this for other users ?
If so, what would be all the criteria ?

 

 

 

[scutters]

On 6/15/2019 at 5:33 AM, Mike_da_Spike said:

This is just for only my cabinet, but is there any interest to have this for other users ?

Yes!!!

On 6/15/2019 at 5:33 AM, Mike_da_Spike said:

If so, what would be all the criteria ?

Deleted tables would be a start (so anything not in the system.xml file, or any media that no longer has a system.xml), with an option to delete media for disabled tables as well would be pretty cool.

[Mike_da_Spike]

Sounds good.

Let me think about it. Too bad I haven't time the next couple of weeks, but after that I can do it.

Are all your video tables matches the name in the xml name ?

I use for myself right now the registry entries I create with this application. But that should match the xml's. 

[scutters]

1 hour ago, Mike_da_Spike said:

Sounds good.

Let me think about it. Too bad I haven't time the next couple of weeks, but after that I can do it.

Are all your video tables matches the name in the xml name ?

I use for myself right now the registry entries I create with this application. But that should match the xml's. 

All my media matches on table name rather than description, but i suppose the xml holds both name & description so could work either way..

[Mike_da_Spike]

27 minutes ago, scutters said:

All my media matches on table name rather than description, but i suppose the xml holds both name & description so could work either way..

Well.. you got description in your xml which contains unicode characters, and files don't can have unicodes.

I will make something which also contains a test mode ( So it will shows what to delete)

Exactly.. I already have that code, because one of my cleaning contains that. 

I will let you known when I have an alpha/beta version of it

 

[Mike_da_Spike]

Version 1.1 uploaded.
In this version you can modify in your INI file a parameter to cleanup unused files (and a debug mode to check which files will be deleted)
The check will be on the file name in the directories and the program check it in the XML if the description is the same. If not (because a version it changed) the file will be marked for deletion

The next directories will be scanned in the media directory for the same PinballX sections that are in the INI file under [PINBALLX] section

Backglass Images
Table Images
Table Images Desktop
Table Videos Desktop
Table Videos
Backglass Videos

Please run this on first time with CleanupDebug = True  !!!!

[scutters]

1 hour ago, Mike_da_Spike said:

Version 1.1 uploaded.
In this version you can modify in your INI file a parameter to cleanup unused files (and a debug mode to check which files will be deleted)
The check will be on the file name in the directories and the program check it in the XML if the description is the same. If not (because a version it changed) the file will be marked for deletion

The next directories will be scanned in the media directory for the same PinballX sections that are in the INI file under [PINBALLX] section

Backglass Images
Table Images
Table Images Desktop
Table Videos Desktop
Table Videos
Backglass Videos

Please run this on first time with CleanupDebug = True  !!!!

@Mike_da_Spike sounds great, i'll try to run it later on today.

Couple of things Just so i'm clear - how does it handle disabled tables/systems, does it just remove media for deleted games or disabled as well?, and i guess it doesn't do DMD images / videos at this time?

I could probably work it out from the debug file - but hey, i'm lazy!!

Thanks

[Mike_da_Spike]

7 minutes ago, scutters said:

@Mike_da_Spike sounds great, i'll try to run it later on today.

Couple of things Just so i'm clear - how does it handle disabled tables/systems, does it just remove media for deleted games or disabled as well?, and i guess it doesn't do DMD images / videos at this time?

I could probably work it out from the debug file - but hey, i'm lazy!!

Thanks

Good one. I use the entries from the registry and dont't put the disabled in it. I can do that.

And about Dmd images/videos: this is a tricky part, because they are based on another name in the xml and could have strange characters in it. I could check for matching wheel names, but to be safe I skipped that part.

Run it ones with the debug=true option and you see which files it will delete.

I'm lazy too with these temperatures 

[scutters]

Looks like it'd cleanup disabled tables by default, which is fine.

Couple of weird errors like;

7/1/2019 9:12:18 AM    ERROR    Error creating [g_objFSO.GetFolder(c:\PinballX\Media\Visual Pinball X Pinsound\Table Videos Desktop)] object, 76 = Path not found
7/1/2019 9:12:18 AM    INFO    Skipping  Table Videos Desktop

Not sure what's going on there, probably something on my set up? Attached log & database file etc for info. 

 

PinballX.ini GenerateUpdateTables.log Visual Pinball X.xml

[Mike_da_Spike]

I will change the errors to informational
this is when i cannot find a directory, I error it (didn't do a check if the directory is there)> thought it was default that all the directories should be there. So you can ignore those, but I will modify this in the weekend (if the temp getting below 20 degrees ...)

Quote

Looks like it'd cleanup disabled tables by default, which is fine.

So you're ok with it ? otherwise need to adjust the code (and need to add the disabled in the registry too and do the filter later)
 

I see that you also have these entry in the log :

skipping [Silver Slugger (Gottlieb 1990)], Because datemodified [6/11/2019 8:54:14 AM] <= [6/11/2019 9:03:39 AM]

Does this means that the modified entries only contains 7 instead of 10 ?
if so, i need to redo the calculation

(I noticed you have some files to be deleted. When I runned it the forst time, it deleted a lot of double videos. because I don't want images, I clear the images folders (DMD, table and backglass) completely
Also .. I want to create my own video's in MP4, so I delete always all the *.f4v files in the video's directory. these are mostly downloaded with gamemanager.exe (but I thought you don't use gamemanager at all)

[scutters]

5 minutes ago, Mike_da_Spike said:

I will change the errors to informational
this is when i cannot find a directory, I error it (didn't do a check if the directory is there)> thought it was default that all the directories should be there. So you can ignore those, but I will modify this in the weekend (if the temp getting below 20 degrees ...)

So you're ok with it ? otherwise need to adjust the code (and need to add the disabled in the registry too and do the filter later)
 

Ahh, ok.. no idea why i don't have all the normal directories, i've never heard of a "Table Videos Desktop" folder before and don't remember having deleted it, but at least i've learned something new today.

I'm ok with removing media for disabled tables, it's easy to recreate if i re-enable them later. I tend to disable rather than delete tables in pinballx and then re-enable for later (or what i consider to be better) versions of a table, and the media needs updating then anyway as i like my table videos to match the loaded table. Might be worth having another flag in the ini to make it optional for others though. 

[Mike_da_Spike]

Quote

I tend to disable rather than delete tables in pinballx and then re-enable for later (or what i consider to be better) versions of a table, and the media needs updating then anywayas i like my table videos to match the loaded table. Might be worth having another flag in the ini to make it optional for others though.

I think That I should skip by default the disabled ones. So if you delete the whole entry in the XML , it can be cleaned too.

[Mike_da_Spike]

Ok. Changed version to 1.2

• It won't delete the disabled video's anymore
• It first check if cleaning directories exists,  if not, just skip these

[scutters]

3 hours ago, Mike_da_Spike said:

Ok. Changed version to 1.2

• It won't delete the disabled video's anymore
• It first check if cleaning directories exists,  if not, just skip these

Thank Mike, will give it a whirl later on if i get chance.

By the way, checked my DMD Images / Videos... their filenames match the table name (i.e. just the same as table images  / videos), might be because i create mine with pbx recorder rather than game manager. So you might be able to include them in the cleanup for some users.

Edit - it ran as expected now, disabled tables skipped in the cleanup

[Mike_da_Spike]

Awesome !

And about the DMD/images/video .... I'm not keen on it to do that. PBX recorder doesn't work for me (all tables are screwed and rotated). 
Gamemanager works fine for me, but uses the description name instead of name within the XML. So not happy to implement it ... But if you want ....

[scutters]

1 hour ago, Mike_da_Spike said:

Gamemanager works fine for me, but uses the description name instead of name within the XML.

Well i guess it would be safe to include on table name and delete those, on set ups like yours it just wouldn't match anything - so no deletes.

It would be harder to document though, something like "..may or may not delete DMD videos/images.." might cause confusion.  I'll leave it to developers discretion and let you decide!! 

 

[Mike_da_Spike]

Ha ! Well I'm not a developer. 

I just want to help the community

Just an ordinary IT admin with a lot of frustration. For an ordinary end user it must be simple Scutters. I think it is a bit dangerous to delete names that are not matching. Maybe it is because I made my own dmd's based on wheelimage name. 

Maybe thats a check I can build in ... let me think about that

[scutters]

1 hour ago, Mike_da_Spike said:

Ha ! Well I'm not a developer. 

All the more impressive!! 

Agree, best to keep it simple - especially where something like deletes are involved that have the potential to mess stuff up.

[Mike_da_Spike]

Version 1.3 updated 

changelog :

Skip removing media files with name "- system -"
New feature in PBX 3.49 to have system wheel and media files