No-Intro - how use them in GameEx

[JK1974]

  fRequEnCy said:

There's a readme?

I thought I was making all this stuff up.

Way to put in some leg work JK1974!

I just overflew the readme first an thought that there is just general information - and that your info was the result of some experiments like I did them.

Because in the last month being in the retrogaming mood, it was quite difficult for me to understand all this GoodTool/No-Intro/TOSEC stuff as I haven´t seen a lot of pages that explain the retrogaming terms and how to work with all those tools and sets - this forum seems to me to be one of the best sources even if it is not directly connected to those tools. Even the so called "clrmamepro guides" are/were difficult to understand if you don´t have a knowledge of some basic retrogaming terms because they are not explained there (which makes those guides quite unusable IMHO)...

That might also have been the reason for not reading the readme all through - because I thought there is no more in-depth information...

So, sorry for bloating up this thread unnecessarily...

  Quote

I think the best and fastest way to support no-intro without parent-clone dats is to just sort on the game title itself and strip the (region) (version) (etc) tags.

...

Matching automatically through a script of some sort you and assume the game title stripped from tags you would get Super Mario World.

So the script could assume that number 2, 3 and 4 should all be put under "Super Mario World". The exception is the 1st one.

If you strip the tags you would get "Super Mario World - Super Mario Bros. 4" which does not match with just plain "Super Mario World".

So this would have to be manually added.

I think that the support of parent-clone dats would be the better (because more error-free) and more standardized solution. Furthermore, it does not look too complicated to me. Just take a look:

	<game name="Lylat Wars (Europe) (En,Fr,De)">
		<description>Lylat Wars (Europe) (En,Fr,De)</description>
		<release name="Lylat Wars" region="EUR"/>
		<rom name="Lylat Wars (Europe) (En,Fr,De).n64" size="12582912" crc="94A1A16A" md5="204A14C2AC815AFEE74B58EF9394708D" sha1="8DE2942E59E31FEA235AD672F6096294D8E7D12E"/>
	</game>
	<game name="Lylat Wars (Australia) (En,Fr,De)" cloneof="Lylat Wars (Europe) (En,Fr,De)">
		<description>Lylat Wars (Australia) (En,Fr,De)</description>
		<release name="Lylat Wars" region="AUS"/>
		<rom name="Lylat Wars (Australia) (En,Fr,De).n64" size="12582912" crc="ED1249A5" md5="A51F94CA0CE8BFE1BAD0192957F0DCE0" sha1="9CBF5087D5684D3B5329630932EB2BFB29AF8511"/>
	</game>
	<game name="Star Fox 64 (Japan)" cloneof="Lylat Wars (Europe) (En,Fr,De)">
		<description>Star Fox 64 (Japan)</description>
		<release name="Star Fox 64" region="JPN"/>
		<rom name="Star Fox 64 (Japan).n64" size="12582912" crc="879EDB89" md5="384EF29B9A1F638EAE21A871C2F1CEA9" sha1="86629DEF8EDBCFD1C6A3DE1C5172B14D1BF2F020"/>
	</game>
	<game name="Star Fox 64 (USA)" cloneof="Lylat Wars (Europe) (En,Fr,De)">
		<description>Star Fox 64 (USA)</description>
		<rom name="Star Fox 64 (USA).n64" size="12582912" crc="363E7EE8" md5="FECDCBBCD1A178A2F4B29B43FACF9F39" sha1="CFF7FB46E79D35F727E4E20CC4C13492ECD2A2D7"/>
	</game>
	<game name="Star Fox 64 (USA) (Rev A)" cloneof="Lylat Wars (Europe) (En,Fr,De)">
		<description>Star Fox 64 (USA) (Rev A)</description>
		<release name="Star Fox 64" region="USA"/>
		<rom name="Star Fox 64 (USA) (Rev A).n64" size="12582912" crc="D71902B0" md5="EF9A76901153F66123DAFCCB0C13CD94" sha1="7D6B2D10DECD487A172FC4C49E44C4F71AB0D703"/>
	</game>

You simply have to read the dat file into memory and parse for "<game name...>". Search in the same line for "cloneof". If it does not exist, make the value of the "name" field a new entry in a db or alternate map file. Otherwise, append the "cloneof" version to the corresponding parent "name" entry. And BTW: I don´t think that you have to check the size, crc, md5 or sha1 - so this would not have to take too much time

.

Parsing them and using some kind of cache algorithm to speed it up the next time it is needed (e.g. check the date of the dat and the number of rom files) would first solve the parent-clone problem and second would not need you to have to merge your no-intro collection - and keep it "intact" for updates etc. (even if this increases the size, but as long as clrmamepro does no solid compression, the community won´t use parent-clone archives).

The major thing will be to put it in a list that works like the GoodTools list, because for detecting the different version, GameEx seems to unpack the archive (why not simply listing the file names?) and then scans for the filenames. This, of course, does not work for the common No-Intro split sets.

And extending detecting (USA) next to (U) is normally just a simple extension of an if-statement

[X1pheR]

  Quote

I think that the support of parent-clone dats would be the better (because more error-free) and more standardized solution. Furthermore, it does not look too complicated to me. Just take a look:

Hi, I agree. Though what do you suggest would be done with the sets which don't have a parent-clone dat available? See below the full list of sets which aren't included in the dat-o-matic and thus don't have parent-close dats available:

Bandai - WonderSwan (NGEfreak)

andai - WonderSwan Color (NGEfreak)

Commodore - 64 (Bigmax)

Commodore - 64 (PP) (Bigmax)

Commodore - 64 (Tapes) (Bigmax)

Commodore - Amiga (SolidSnake, Connie)

GamePark - GP32 (SolidSnake)

Nintendo - Nintendo Entertainment System (BigFred, SolidSnake)

Nintendo - Game Boy (NGEfreak)

Nintendo - Game Boy Advance (e-Cards)

Nintendo - Game Boy Color (NGEfreak)

Nintendo - Super Nintendo Entertainment System (NGEfreak)

Nokia - N-Gage (SolidSnake)

Tiger - Gizmondo (SolidSnake)

Now most I don't play except for the nintendo sets. I play these very often. So it would be nice to have these displayed in goodmerge style also.

So besides parent-close dat support there would be another added support for these like the way I explained.

Also what is your take on emumovies support? These are all named to the USA sets. But if you use parent-clone sets then it should still be able to find the video.

Otherwise one would have to rename the videos I guess? Which you don't want because of future updates to the sets.

[X1pheR]

I checked the No-Intro forums and it seems that several parent-clone versions are already completed but just not included in the DoM.

For example:

NES

SNES

[wahoo]

  JK1974 said:

Hi,

I have used GoodTools sets before in GameEx, and this is working great: Only one title in the gamelist, all "subversions" in the detailed view including language flags to the left.

However, I don´t need all the hacks, overdumps etc., just the "originals", so renaming to No-Intro would be the way to go. And it seems to me that the bigger part of the GameEx community uses No-Intro sets in the meantime.

However, with my setup, No-Intro looks more uncomfortable, as if GameEx does not "support" No-Intro: Most games are listet several times in the game list (because the No-Intro sets are not merged, so you get the languages in the "main list"), and of course, the detailed view does not work as it does with GoodTools set.

To get the game list cleaned up, I played with the filter options, but those don´t work as they just filter the sets inside the archives, not the archives themselves. So, to get e.g. only the European releases, would I really have to move all US and JP roms to a different folder?

I also converted a GoodTools set to No-Intro and used GoodMerge to convert it back. Of course, this worked and gave me the result I expected, but this would also remove newer No-Intro roms from the set. Furthermore, GoodMerged No-Intro sets are not really common. With screenshots following the No-Intro convention, this also would not help.

So, did I oversee some options, or are there some more sophisticated ways to get No-Intro sets working as "clean" as GoodTools sets? Or do we simply have to wait for further GameEx developments?

Thanx a lot in advance for your suggestions.

Lots of good stuff being discussed in this topic. Sorry to repeat the information already discussed in this topic and to repeat my feature request. I haven't heard any feedback so I'll try one last time.

GameEx already "supports" No-Intro along with GoodTools and TOSEC including the databases being developed.

Here are three options for using no-intro that are being discussed in this topic:

Option 1: Using them in merged mode. GameEx already supports this. How they get merged is outside of GameEx. There are some good suggestions posted in this topic discussion out here how to do this without having a utility like GoodMerge for no-intro sets.

Option 2: using a flat list, removing duplicate copies where they exist, keeping the preferred copy of available versions. I assume this solution will always reside outside GameEx?

Option 3: using a complete flat set "as-is" there will be multiple versions of some games.

Options 1 and 2 address the issue of trying to skip through multiple languages to find the version you want, reducing the amount of apparent duplicates in the no-intro list. Option 1 and 2 have their pros and cons already discussed in this topic involving manipulation of the sets.

Option 3 is for those that want to leave the no-intro sets intact, which is what most are probably doing. Multiple web sites provide snaps, videos and titles for the no-intro name convention so by leaving sets intact no renaming necessary. I assume that most are using this option also enabled the option to remove all text within brackets when displaying the game list. This configureation option cleans up the title names, but will result in apparent duplicates.

Implementing a simple feature in GameEx could enhance option 3 and make locating the desired language version of the game more quickly and reducing the apparent duplicates (because the language flag will indicate it is actually another version.)

The feature request is to add a language flag next to the game in the main game list based off of text that is in parenthesis like it does inside a merged set. I assume this would be easier to implement than trying to accomplish option 2 inside GameEx.

This feature was mentioned in a post from igelkotten:

http://www.gameex.info/forums/index.php?s=...ost&p=53730

And it is also in a feature request I submitted:

http://www.gameex.info/forums/index.php?s=...ost&p=50823

Here are some added bonuses with the language flag feature:

Works with or without roms being zipped (registered or unregistered)

Enhances the process of navigating through non merged sets regardless of the naming convention GoodTools/No-Intro/TOSEC/(Others) as long as it consistently used common text for indicating the language version such as (U) (USA) etc..

Thoughts?

[chriss]

wahoo, thanks for your quick summary! This thread really throws in a lot of interesting ideas!

I agree that for real collectors the only way will be to leave the rom sets untouched. But I also think that there wont be a lot of interesting updates to these sets (i mean rom-wise) so I don't see a reason for updating my no intro sets anymore. I'm pretty sure that all my friends who are looking for snes roms on my arcade will find the ones they know from their childhood.

Thats why for now I only keep the USA roms mostly because every game then has a video from emumovies with it.

Out of convenience I wouldnt be a big fan of option 3. Going through a list with thousands of games and having the same name appear over and over again seems a bit confusing to me. So the only real update to my set as it is right now would be to have available languages in a submenu (the way JK showed it in his screenshot). The main list would only consist out of one game title and then you have language icons in the submenu.

But I think for now I just stick with having all the roms that were released for the US market. I guess I'm a lousy collector but thats really all I need for some quick retro gaming pleasure!

[Tom Speirs]

  chriss said:

Going through a list with thousands of games and having the same name appear over and over again seems a bit confusing to me. So the only real update to my set as it is right now would be to have available languages in a submenu (the way JK showed it in his screenshot). The main list would only consist out of one game title and then you have language icons in the submenu.

I tend to like this idea best right now, where gameex filters the list distinctly, and then acts like goodmerge does on the info page. I guess we'd just need an emulator setting to turn this on/off. When the XMLs that were discussed become mature, we can move to that.

[JK1974]

  Da Killah said:

Hi, I agree. Though what do you suggest would be done with the sets which don't have a parent-clone dat available? See below the full list of sets which aren't included in the dat-o-matic and thus don't have parent-close dats available:

Bandai - WonderSwan (NGEfreak)

andai - WonderSwan Color (NGEfreak)

Commodore - 64 (Bigmax)

Commodore - 64 (PP) (Bigmax)

Commodore - 64 (Tapes) (Bigmax)

Commodore - Amiga (SolidSnake, Connie)

GamePark - GP32 (SolidSnake)

Nintendo - Nintendo Entertainment System (BigFred, SolidSnake)

Nintendo - Game Boy (NGEfreak)

Nintendo - Game Boy Advance (e-Cards)

Nintendo - Game Boy Color (NGEfreak)

Nintendo - Super Nintendo Entertainment System (NGEfreak)

Nokia - N-Gage (SolidSnake)

Tiger - Gizmondo (SolidSnake)

I completely agree with you, but I believe that this is some kind of work-in-progress.

If it takes too much time until dat-o-matic is ready, we could try to build this parent-clone sets on our own by comparing the filenames as it has already been suggested. I haven´t tried to generate a *.dat file yet, but I think that if such a half-automatically merged set exists, it might just be a few clicks in clrmamepro to generate a parent-clone *.dat that can be delivered with the GameEx download package. Of course, we can give this parent-clone *.dat back to the no-intro community - maybe this and the support within GameEx will speed things up.

[X1pheR]

  TSpeirs said:

I tend to like this idea best right now, where gameex filters the list distinctly, and then acts like goodmerge does on the info page. I guess we'd just need an emulator setting to turn this on/off. When the XMLs that were discussed become mature, we can move to that.

That would be excellent! I'm new to GameEx and am still trying it out but this is something I missed right away. As wahoo said he already requested it and it was also already mentioned by somebody else in the feature request topic. But to almost all no reply it given as in if it will be included as in worked on. Since you like the idea. Will you be implementing it? If so and any testing has to be done I would be glad to volunteer

Some suggestions if you don't mind which I was thinking of:

• For almost all no-intro sets which are not in the DoM users have already created parent-clone dats and xmdbs. I tried out the SNES and NES and they work 100%. Just as a notice.
  
• If you would let GameEx generate the Parent-Clone lists by itself by my understanding for no-intro you could just take the game title untill the first ( sign, right?
  
• Maybe also add support for this to the map files so GameEx does not have to generate this on every start. This way users could also create their own parent-clone map files. This way somebody could extend the support to not only no-intro but also other sets like Daphne or ScummVM. For Daphne most games have several versions (remastered, HD, original, etc). For ScummVM also in the form of DOS, Amiga, Mac, Windows, Floppy, CD, etc. It would be great to be able to display these sets in the same goodmerge way.
  
• Already mentioned but adding the country flags right away would also be nice. Maybe even nicer to let the user add support on their own. Creating own tags related to icons. This way for the (flags) with Daphne, ScummVM, etc sets one could also create icons for this.
  
• With this implementation GameEx should ofcourse still recognize media set files (cartridge, boxart, videos from emumovies, snaps, etc) which are in the no-intro naming and use them for the goodmerged game titles. Because as with the rom sets you also don't want to rename your media files. But again, matching a media file to the name untill the first ( sign should do that right...
  

Sorry to post so many items. Can't help getting excited for this feature

  JK1974 said:

I completely agree with you, but I believe that this is some kind of work-in-progress.

If it takes too much time until dat-o-matic is ready, we could try to build this parent-clone sets on our own by comparing the filenames as it has already been suggested. I haven´t tried to generate a *.dat file yet, but I think that if such a half-automatically merged set exists, it might just be a few clicks in clrmamepro to generate a parent-clone *.dat that can be delivered with the GameEx download package. Of course, we can give this parent-clone *.dat back to the no-intro community - maybe this and the support within GameEx will speed things up.

Indeed. Like mentioned above I've tested some and they work 100% though. So it seems that I believe all are already 100% working. They just haven't been added to the DoM yet. But I believe Tom nonetheless likes to let GameEx handle it by itself. This could have the added bonus as mentioned to maybe be able to edit them and give support to sets like Daphne or ScummVM.

[Tom Speirs]

Ive got GameEx doing it now here. It works very well for flat goodsets also. It trims a goodset down by about 75% and a NoIntro by about 18%.

[X1pheR]

Damn! You implemented it that fast? Cool...

Did you perhaps read my suggestions? Was wondering what you think of them...

[Tom Speirs]

  fRequEnCy said:

Region flags will be missing cuz regions are listed different between no-intro and goodtools. Tom would have to support the no-intro region format as well. So perhaps someone should put these features in the request pinned topic so that Tom will be more likely to see them.

Ive got it supporting the flags for No-Intro now too.

[fRequEnCy]

  TSpeirs said:

Ive got it supporting the flags for No-Intro now too.

Nice Tom!

[wahoo]

  TSpeirs said:

Ive got it supporting the flags for No-Intro now too.

This is great news! Thanks Tom!

[JK1974]

Simply unbelieveable, I am looking forward to it...

[bkenobi]

Damn it! That means I've got to go rename my sets now

Thanks Tom!

[X1pheR]

Flag supported also already. Thumbs up! Can't wait to see it

How will media files be matched if the way you implemented it now? Do media files now have to be renamed goodmerged style as in game title only without flags.

Or can it use any media file which matches with one of the full rom name (with flags) and use that as the main media for the goodmerged title?

Am I making any sense? Sorry, I'm dutch and it's late

BTW, did you see the suggestions I posted a few posts back?

[Tom Speirs]

If you have the extra snap matching turned on it searches for a match without the bracketed text already if it cant find an exact match. Also it searches based on the file name so what's displayed in GameEx is irrevelant. So by default it will show the snap for first game in the rom folder found alpha numerically if using this new merge feature. So if Europe and USA exist Europe will be shown as e is before u. On the info page it will match the snap for each variation. It does not do anything else clever for now.

Yes, some interesting suggestions.

[X1pheR]

Nice. So with no-intro (or goodsets) sets being able to be displayed as goodmerged it will still match against your untouched media files. That's excellent. This is gonna be good Hopefully the displaying of media files in goodmerge mode will not be slowed down a lot since for every listing it doesn't match so it searches.

Maybe it is in fact better to just audit all media collections as goodmerged. This way it will match right away with the goodmerged game title but also slim down the size of all your media collections as well. Also with using sets like emumovies videos you only get USA sets. So by just renaming these sets to remove the (USA) they are in fact goodmerged right away

And thanks. Hopefully the suggestions will be interesting enough to implement

[Tom Speirs]

The way I see it I am just having a little go at making it a bit easier and a little quicker, like with a lot of the features. End of the day if you want the perfect and most importantly accurate media matching and perfect merged sets people are going to have to do a little work. The other bonus is that GameEx wil support it if you do

Anyway, I think ill release it shortly, so you should be able to get a better idea, might as well get it out there while the irons hot. Seems like theres been quite a bit of innovation in this thread. I never heard of anyone Goodmerging NoIntro before!

NoIntroMerge © GameEx Forum Members!

[fRequEnCy]

  TSpeirs said:

The way I see it I am just having a little go at making it a bit easier and a little quicker, like with a lot of the features. End of the day if you want the perfect and most importantly accurate media matching and perfect merged sets people are going to have to do a little work. The other bonus is that GameEx wil support it if you do

Anyway, I think ill release it shortly, so you should be able to get a better idea, might as well get it out there while the irons hot. Seems like theres been quite a bit of innovation in this thread. I never heard of anyone Goodmerging NoIntro before!

NoIntroMerge © GameEx Forum Members!

Sounds good Tom! Thanks! Can't wait to see the new release.

[X1pheR]

  TSpeirs said:

The way I see it I am just having a little go at making it a bit easier and a little quicker, like with a lot of the features. End of the day if you want the perfect and most importantly accurate media matching and perfect merged sets people are going to have to do a little work. The other bonus is that GameEx wil support it if you do

Anyway, I think ill release it shortly, so you should be able to get a better idea, might as well get it out there while the irons hot. Seems like theres been quite a bit of innovation in this thread. I never heard of anyone Goodmerging NoIntro before!

NoIntroMerge © GameEx Forum Members!

You sure are! The innovation has just begun I guess. With no-intro's Dat-O-Matic you now have parent-clone dats and xmdb files to go with it. With that making it possible to create a GoodMerged set with the GoodMerge program in just a few clicks. ClrMamePro still doesn't support solid archives yet. But when it does (and from what I've read it is going to happen) I think more and more will convert the sets. Correct me if I'm wrong but I only see upsides as to no-intro sets getting GoodMerged.

+ One listing for every game in stead of a full list with all versions.

+ Much lesser filesize being the different versions put in one solid archive (which only GoodMerge does at the moment).

+ Resulting in also much lesser sized media collections as you only need one media file for the goodmerged romset.

Downside is that no frond-end supports it yet, untill now I believe you are the first one to support it.

Maybe I'll even GoodMerge my romsets now with the support being added now. Correction; I'll No-IntroMerge them :P

[Tom Speirs]

Thanks for the education. It seems really it already supported it other than the flag icons, so I cant really take credit for doing something new in a front-end. I think the automatic merge is nice though. Might be nice for MAME too. I personally like to have the list of clones in MAME, but only when the actual game name is different. Hence why GameEx has always filtered the MAME list by default.

[X1pheR]

Education? Sorry, really didn't mean it in that way. I was just spilling my thoughts about. Also for other people getting to read this topic I guess. Guess I just talk to much

I'm currently converting the SNES set with goodmerge. I'm curious how big of a difference the sizes will be. Will post results tomorrow. Going to sleep now. Goodmerge reports also that it will take 2,5 hours to convert!!! Damn

[igelkotten]

  TSpeirs said:

Ive got it supporting the flags for No-Intro now too.

Great news, Tom. And the new Parent/Clone support too.

Looking forward to the next release...