The GameEx Database Project

[headkaze]

I do a scrape when I start a new db. I will not re-scrape until I have completed quite a few.. I will never get it done otherwise

Good point

The SEARCH was not for an online search, but for the SEARCH&REPLACE function.

Follow me here....

At the moment S&R checks for the FULL record and replaces with the FULL amendment - This is exactly as needs to be, perfect.

But a standard search (within S&R) needs to check for "within" a record, so a search of "ACME" will find "ACME Sausage Factory", "The ACME sausage Factory", etc. Using this I could find orphaned entries where a dat has used a slightly different name.

THIS MUST NOT affect S&R..

Implementation:-

Clicking "Replace" - Search for FULL matches and replace with FULL matches.

Clicking "Search" - Search for PART matches withing fields and display results only.

I getcha now, and that will be easy to add and good idea!

Ok, now down to Scrapes again.

When you scrape, new data is entered into the current db. You have no way of knowing which are new and in this way the pub,dev, and name caches will become muddled again. Can a new scrape use a triangle (example) as an icon (instead of the coloured circle) in the game list, and remain (even when db saved) until that record is edited? This way, you will know what needs to be checked at all times. All the db's need is an "Edit Flag". Set to 0 by default and change to a 1 to mark a modified entry. Do ya catch me drift!!??

Actually I already made them go blue colour but are you really colour blind? (note I'm using the correct spelling for colour). And the status of blue means "newly scraped" and it does save that status. You just have to set the status colour when you edit it to change it. If you're really colour blind (sometimes I don't know when your being serious) then what colours can't you see? And if so I'll have to find a friggen triangle shaped icon (yay!).

In the games list of an open dat, can the devs and pubs of non-matching entries (dev is not in cache list, etc) be displayed in bold or similar (remember that I am colour blind). This way, a scan of the list will show me edits that are needed. Not sure how to implement this because any dev with a single entry in the cache would be marked in bold. Perhaps something could be implemented with the Modified flag? Or is this a unusable idea..

Again I don't quite understand, last post you said you liked playing with the red "blushing" entries. Can you see red or not? I'm not even sure if I can set text bold in a listview.

Also there is no way to do this really because when you scrape as a game is added it will add all it's data to the global cache. So you will need to go through each entry manually and fix any that have added new name/pub or dev entries.

Can the column headings be swapped around and remembered. So I can have the dat names at the back and the year after the name if i need to. On lists with large names the dat names are off the right of the screen In this event, I could move Name to the back so the first 3 are the dat names for easy checking etc... Or perhaps I could move the dats behind description so I have a better view of the data for a game?

Column re-ordering is easy to add, but remembering the order could be a bit tricky. I'll look into it.

Do you like all my minus signs?

What minus signs? Didn't I tell you I'm minus sign blind?

I will update the tosec/good/nointro at some time, but am quite scared of using updated ones as I have removed and amalgamated so many entries that an "Add missing" could really mess a dat up. Try having a play with the 7800 or 5200 for an example of "Add Missing".

You also get multiple entries again...

True dat. Anyway I might have a poke around today and see if I can find any updates.

More minuses and Thanks for ALL your work HK..

Where? Are you teasing me?

[greatflash]

Actually I already made them go blue colour but are you really colour blind? (note I'm using the correct spelling for colour). And the status of blue means "newly scraped" and it does save that status. You just have to set the status colour when you edit it to change it. If you're really colour blind (sometimes I don't know when your being serious) then what colours can't you see? And if so I'll have to find a friggen triangle shaped icon (yay!).

Yes, I am colour blind in R G and B, more in some than others. The red I can see (just - it is better on the monitor at home than the laptop at work). But, a triangle would be really nice and make me happy

Again I don't quite understand, last post you said you liked playing with the red "blushing" entries. Can you see red or not? I'm not even sure if I can set text bold in a listview.

Also there is no way to do this really because when you scrape as a game is added it will add all it's data to the global cache. So you will need to go through each entry manually and fix any that have added new name/pub or dev entries.

But, if when a scrape is made, each NEW entry gains a flag to say it is new, then your 'cache routine' would notice the flag and NOT add them to the cache. Only when the entry is modified would the flag be removed. Would this not work. The initial trawl through the dev and pub lists took over 40 hours (with some dev checking also), If I have to go through these everytime I do a scrape - nightmare. I know it will not take as long, but it is still a massive amount of dev and pubs to check through..

Column re-ordering is easy to add, but remembering the order could be a bit tricky. I'll look into it.

It needs not remember them, just not to re-order each time an edit is made.

What minus signs? Didn't I tell you I'm minus sign blind?

There! Put a hand over one eye and squint!

True dat. Anyway I might have a poke around today and see if I can find any updates.

there must be a way to stop and "add missing" messing up the db?

Where? Are you teasing me?

Yes - I was ---------

[headkaze]

DBM Version 1.73 Released!

I'm really excited about this release and I'm sure you will like this one too Flash

- Added column re-ordering and it will save/restore column positions

- Changed blue icon to be a... blue icon with a plus sign in it (I call it the "New" icon). Hopefully that is clear enough now. Also text is never red anymore; only inverse colors now so it should be really easy to see hilighted text

- All new entries are now marked with the "New" icon. No data is added to the cache when it is in this state. When you browse the game list, items that are not in the cache are inversed, so you can see exactly what games you need to look at. So everything should be nice and clear now

- Search & Replace; when you click on Search it will match any portion in the search term (and is not case sensitive). Replace still does an EXACT search & replace of the search term.

- Your gonna love this one.. you can now edit games directly in the search & replace results list, just double click a game in the list to edit it! I believe this will be a well used feature.

Note that all new entries (including those added using "Add Missing", adding a new game, scraping etc.) are now marked with the "New" icon (blue icon with a cross). And the data will not enter the global cache until you edit that game and change it's status to Red, Amber or Green. This is true even if you save a database and leave a game in the "New" status.

[Tom Speirs]

Just wanted to say thanks HK. Your making me feel like Im being lazy

We should benefit from all this in the next release of GameEx though.

[headkaze]

Just wanted to say thanks HK. Your making me feel like Im being lazy

We should benefit from all this in the next release of GameEx though.

Hehe no worries! It's all about sharing the workload

EDIT: Found a bug in the S & R editing. Now fixed in 1.74

[greatflash]

DBM Version 1.73 Released!

I'm really excited about this release and I'm sure you will like this one too Flash

- Added column re-ordering and it will save/restore column positions

- Changed blue icon to be a... blue icon with a plus sign in it (I call it the "New" icon). Hopefully that is clear enough now. Also text is never red anymore; only inverse colors now so it should be really easy to see hilighted text

- All new entries are now marked with the "New" icon. No data is added to the cache when it is in this state. When you browse the game list, items that are not in the cache are inversed, so you can see exactly what games you need to look at. So everything should be nice and clear now

- Search & Replace; when you click on Search it will match any portion in the search term (and is not case sensitive). Replace still does an EXACT search & replace of the search term.

- Your gonna love this one.. you can now edit games directly in the search & replace results list, just double click a game in the list to edit it! I believe this will be a well used feature.

Note that all new entries (including those added using "Add Missing", adding a new game, scraping etc.) are now marked with the "New" icon (blue icon with a cross). And the data will not enter the global cache until you edit that game and change it's status to Red, Amber or Green. This is true even if you save a database and leave a game in the "New" status.

Thanks HK.. (Again)

Really looking forward to having a play today.

Love the idea of the direct search editing, this could be a real time saver and at the very least - a great way to maximise the cohesion of the data.

Now that I have got EVEN more professional tools, is there any chance of a pay rise - I was thinking of a %50 rise and overtime pay for weekends

I will let you know how I get on tonight when I get back from work HK. Will be 8pm GMT (you know - real time )

Thanks again..

PS. Have you noticed how lazy Tom is getting?

[starbuck]

Morning all,

Database work is fantastic. You both deserve a much deserved THANK YOU.

I'd like to throw out a request I Made of GameEX a while back, that i think might now be doable, to help both the users and to better build the database. Have the Database put in all the games in the user game list from the database. If you have it, turn the text green or use a symbol. I f you have a game that is not in the database turn it yellow and then we will know to contact you guys about a possible needed addition. If you don't have it turn it red, and we will know we need to get that game.

I don't know of a frontend that has this feature so this would be a first.

[greatflash]

Morning all,

Database work is fantastic. You both deserve a much deserved THANK YOU.

I'd like to throw out a request I Made of GameEX a while back, that i think might now be doable, to help both the users and to better build the database. Have the Database put in all the games in the user game list from the database. If you have it, turn the text green or use a symbol. I f you have a game that is not in the database turn it yellow and then we will know to contact you guys about a possible needed addition. If you don't have it turn it red, and we will know we need to get that game.

I don't know of a frontend that has this feature so this would be a first.

You can do something similar.. You will have to follow me here..

In the emulator config.

--------------------------

Make sure the correct db is selected

turn on "database naming"

make sure "remove text in brackets" is OFF

When you go to the list of games in GameEx, the ones that are in the db will have the bracketed text present, eg (1991) [hack] (Univesal) etc.

The games that are in the db will look normal.

This is really only of worth with db's that are complete (see first page of this topic). Though, at the moment I really could do without "game XXX is not in the db posts". Most games that are not matched in the complete db's are the result of incorrect file names, though I am sure that there will be some missed and am doing my best to make sure there is not. Things that are missed at the moment are a lot of the pd, demos, and hacks on the larger databases. These will be added later, but my main concern it to get as many actuall GAMES in as possible.

The problem with adding a function to show all roms in the db and colour missing ones different (apart from perhaps messing up some themes colour schemes) is that this takes GameEx more into the realm of rom manager, something that i am sure Tom does not want.

Saying that, a seperate tool that used the db's to check against your roms and offer a naming function may be something worth looking at when the db's are complete (this could be a while )

Thanks for the input!

[starbuck]

Thanks for the reply

Understand what you are saying. Did a number of beta tests for a number of front ends in the past. As a result I seem to have a lot of hacks, prototypes, etc of games for a lot of systems that don't seem to show up anywhere. As you have noted no one manager seems to support all the systems or all the games. But I'm one of those crazies who would like to list all the games ever made for a system. Always neat when showing off your cab, a game that no one has ever heard of or played.

[greatflash]

Thanks for the reply

Understand what you are saying. Did a number of beta tests for a number of front ends in the past. As a result I seem to have a lot of hacks, prototypes, etc of games for a lot of systems that don't seem to show up anywhere. As you have noted no one manager seems to support all the systems or all the games. But I'm one of those crazies who would like to list all the games ever made for a system. Always neat when showing off your cab, a game that no one has ever heard of or played.

You must be mad!!

As I say, The db's support ALL games and prototypes. I personally dislike hacks (except english language hacks - when 100%), hacks are the bastard sons of hell and should all be destroyed . Why anyone would want to play super mario world with mario replaced by sonic, or another version of Adventure on the VCS with more unplayable mazes, I will never understand. Hacks have no value, and as someone who really appreciates the history of the games - they are worthless.

Now, public domain and new games for old systems are a different matter. Though 90% of the games are RUBBISH, there are a number of games that really shine, games by John Dondzilla, Ken Siders, Arnauld Chevallier, etc are worthy of any collector. These are in the db's for the most part. A lot of the db's are totally complete and actually contain prototypes that are known of but have never been found or released.

Demos can be worthy of inclusion for technical reasons, and are included in smaller db's like the Atari 2600, 7800, Intellivision, Vectrex, etc.

At the moment, larger db's will not have demo's or hacks. these will be added at a (much) later date for reasons of time. I really want to get the games done. As well as that, the only real data that you may get back for a demo or hack is the year. There is precious info available about who did them or what they are, so in a way - the inclusion is rather pointless.

As I say, the db's will contain (and do) ALL known games (or official releases), and all known prototypes. This is really what it is all about, the historic collection of the GOLDEN age of games.

Phew..

Thanks for your interest and I hope this helps to explain the db's a bit.

PS. Still think you are mad keeping all the hacks. Do a directory search for [hack] and (hack), select all, press delete. Then have a little dance and be pleased that your system has been cleansed!! :)

[greatflash]

HK:

Guess what? Betcha can't... Go on, try!

Yep, that's correct - "I have had a brainwave".

Ok, this is a new function for DBM called "Directory Scrape". I think you can work it out already

Take the C64 goodbase set. This is currently at V5 though the dat is only at V3. Wouldn't it be great to scrape my c64 rom directory and add all the games that are missing? I have the complete v5 set which is much larger than the V3.

Take the Saturn or Dreamcast sets. These have no real dat (small iso ones that is all), but all the rom (iso) names follow a set pattern. Mine are named using the Tosec rules. It would be great to scrape the saturn directory to add all the new games that are not in the dat or that I have not manually entered.

Take the Naomi sets that are now turning up. I could scrape that directory and quickly add the new ones to the db ready for editing..

This would be fantastic for any db that does not have a dat!!!!

Good idea?

On the scape screen requesting directory it would need a few options. I.e.

Add Data to - "Goodname/Nointro/Tosec"

Select witch on to add the data to, ie the name of the files found. This would be non destructive and only add new records.

this would also add the name to the "name" field.

Strip Brackets - "yes/no"

This would indicate if brackets should be removed from entries. (this is handy for c64 mostly)

Of course this would also check against existing data for repetition and mark new entries the same as a scrape or add missing.

When entering names to the "name" field, the subject of "xxx, the" and "the xxx" should always be entered using the later. I try to structure all my names to use "the" first when needed as this creates a tidier list.

That reminds me, need to go through all the publishers and devs (AGAIN) and rename to put "the" at the front, this will also find doubles for me.

Sorry if this request sounds more like an instruction

OH, quick option.

When the pop-up "XXXX has been added to the xxxx cache", is there any chance I could have a warning first with an option of ok or cancel?

All the best my overworked friend..

I will try not to bother you anymore.

[headkaze]

DBM Version 1.75 Released

- Now prompts before adding to global cache

- Changing status from the GameList now prompts before adding to global cache for "New" games

- If you say no to adding any field to the global cache the game will remain in "New" mode. You must resolve the global cache before you can change it's status from "New".

- Added new feature: Tools->Folder Scan. Scans a folder adding file list to GoodTool/NoIntro/TOSEC field. Can you test this feature for me Flash?

Phew another large update but I did really like that Folder Scan idea so had to add it. Also fixed a couple of bugs at the same time. Let me know of any more bugs.

[greatflash]

DBM Version 1.75 Released

- Now prompts before adding to global cache

- Changing status from the GameList now prompts before adding to global cache for "New" games

- If you say no to adding any field to the global cache the game will remain in "New" mode. You must resolve the global cache before you can change it's status from "New".

- Added new feature: Tools->Folder Scan. Scans a folder adding file list to GoodTool/NoIntro/TOSEC field. Can you test this feature for me Flash?

Phew another large update but I did really like that Folder Scan idea so had to add it. Also fixed a couple of bugs at the same time. Let me know of any more bugs.

Thanks again HK.

you are on a roll here

Getting through the SNES. It is amazing how many Japanese games have no data on them anywhere. Spent over an hour on one game and all I could work out from a Japanese translation was that it was a "managament sim based around a Japanese newpaper". Not very descriptive. Must order my "Learn Japanese in 2 weeks" audiobook.

Does not sound like much, but am into the "B"'s and I could not find any bugs, so well done for finding any.

Thanks as always

FLASH

[bkenobi]

The B's? Wow, you haven't made progress until you reach S!

Nintendo really likes using "Super", so that's around half the games!

[greatflash]

The B's? Wow, you haven't made progress until you reach S!

Nintendo really likes using "Super", so that's around half the games!

I know

Thing is, when you start a fresh db, the first thing to do is go through it all and try to match orphaned data. This takes a while, before even starting. The problem with the whole nintendo thing is the sheer amount of Japanese games (and these hardly ever start with "super"), so - in the end it pretty much evens itself out...

in that case you are wrong But i will give you one point for effort

[bkenobi]

Damn Japanese games! Who plays those anyway <turns to look at Dreamcast collection> Oh, never mind

[greatflash]

Damn Japanese games! Who plays those anyway <turns to look at Dreamcast collection> Oh, never mind

Ha Ha... I wish no one did. The db would be done 10x quicker.

[greatflash]

HK,

I have posted the C64 xml as I need your help.

In the name field, many (most) of the names have the bracketed goodbase numbers after them. This is the after effect of add missing with brackets on. I needed the brackets in the goodname part but not in the name part.

If there any way you can run a quick script to remove them (leaving them in the goodname). I tried but between red wine, excessive ebay bidding, and the chimes of 4am - I failed...

Ps. Won an Acorn Atom (boxed) for £175 ($274 or $407 AUD) on Ebay last night... That will be a nice one to add to the gallery (and a Sharp MZ700).

Ps. Done pictures of PSone and original Playstation ready to add to that gallery that my esteemed friend has still not found the time to register on..

So, pretty please on the c64 xml.

EDIT: Another option is Wildcard matching in search and replace. This would be a good idea if partial replace only occured in S&R when wildcards are used and remained a strict replace without them.

I could them search for "*(*)" in the name field and replace with "". Hm... Problem is this would replace things like "Zybex (preview)" with "Zybex", and would also to ALL db's rather than be confined to just the c64 one.

Wildcards could be of use when it comes to compilation packs and the occurence of multiple developers seperated by ", ".

Oh well, just the mutterings of a man with a hangover. I don't drink, but had half a bottle of wine last night... big mistake!! :)

_PC__Commodore_64.7z (

[Tempest]

Damn it Flash! I was going to take a look at renaming them, but you password protected the file! Oh well, I just wanted to contribute a little bit...

[greatflash]

Damn it Flash! I was going to take a look at renaming them, but you password protected the file! Oh well, I just wanted to contribute a little bit...

My boss "HK" told me to...

Anyway - I like to keep him busy, but... Thanks anyway Tempest... (I still won't let you off for failing me with the Channel F though )

Cheers!

[Tempest]

No problem, I know HK wants to keep a lid on this stuff. I've been dabbling in AHK, and renaming is easy to do. I've enjoyed it so much, I've renamed my cat, my wife, my wife's "Good Stuff", pretty much everything around the house...

[greatflash]

No problem, I know HK wants to keep a lid on this stuff. I've been dabbling in AHK, and renaming is easy to do. I've enjoyed it so much, I've renamed my cat, my wife, my wife's "Good Stuff", pretty much everything around the house...

That must have been one hell of an AHK script.

I just used F2 on the wife and pressed DEL on the dog. Sadly I had accidently CTRL clicked the plasma TV as well

I checked the recycle bin, but it was just full of cardboard and plastic bottles

[headkaze]

Version 1.76 Released

- Wildcard matching for Replace

- Data->Name->Remove Brackets added

[greatflash]

Version 1.76 Released

- Wildcard matching for Replace

- Data->Name->Remove Brackets added

Thanks HK..

will have another play tonight.

I will upload all new XML's when I post Tom the mdb's next week to the usual place..

[headkaze]

Just a bit more info on the wildcard matching you can use either a * or a ? character in the search. Wildcards do not work on a normal search as that just matches any part of what you type (non case sensitive). The wildcard matching is case sensitive.

Let me know how the "Scan Folder" thing works, have you had a chance to test it yet?