Arcade Cabinet


Valid XHTML 1.0!

Valid CSS!


News

CAESAR

Forum

Dats

Tools

DatFAQs

HowTo

Links


X-Arcade

X-Arcade


Nebula Dat

Last updated: 17th July 2004

Overview

Although Nebula can generate a RomCenter data file, there are a few good reasons for creating an alternative one:

  • The CAESAR database needs year and manufacturer information to be included (neither of which are supported by the RomCenter format).
  • SHA1 information can be added for ClrMamePro.
  • The Nebula drivers contain some mistakes that make accurate ROM management difficult/impossible (due to CRC conflicts).
  • The data file generated by Nebula is not completely accurate for some reason. Amongst other things, it contains some unneeded ROMs and also has some ROMs missing!

DatUtil and MAMEDiff can understand Nebula drivers (i.e. the contents of the 'romdata' directory) but the facilities are only suitable for games that have 'romdata' drivers. It is therefore easiest to maintain a Nebula data file as two separate files; one containing the 'romdata' games and the other containing the remaining games. Each of the two files can then be validated using the techniques described later on in this document.

There are a a variety stages involved in updating the Nebula data file :

  • Identifying the changes in a new version of Nebula.
  • Identifying and fixing problems in the Nebula drivers (necessary if a data file is to be created that matches the drivers).
  • Validation of the data file for 'romdata' games against the 'romdata' drivers themselves.
  • Validation of the data file for the remaining games against the Nebula generated data file.
  • Combining the two data files ('romdata' games and the remaining games) then adding SHA1 information.
  • Building ROMs and checking them within Nebula.
  • Just for interest, checking the accuracy of the RomCenter data file generated by Nebula itself.

Note: The batch scripts in this document use ports of the Unix commands cat, rm, and sed. When trying to get the batch scripts to work, cat.exe, rm.exe and sed.exe must be in the path (e.g. c:\windows).


Identifying Changes in a New Version of Nebula

Identifying changes in a new version of Nebula can quickly be achieved using MAMEDiff. Firstly, the 'romdata' directories of the two versions must copied into directories such as 'Nebula v2.23d' and 'Nebula v2.24d'. After that, the following batch script can be used to compare them:

@ECHO OFF

SET NEBULA_VER_PREVIOUS=v2.23d
SET NEBULA_VER_CURRENT=v2.24

REM --- Concatenate all previous the drivers into one file (%NEBULA_VER_PREVIOUS%.tmp)

cat Nebula?%NEBULA_VER_PREVIOUS%/romdata/*.dat Nebula?%NEBULA_VER_PREVIOUS%/romdata/*.DAT >%NEBULA_VER_PREVIOUS%.tmp

REM --- Concatenate all current the drivers into one file (%NEBULA_VER_CURRENT%.tmp)

cat Nebula?%NEBULA_VER_CURRENT%/romdata/*.dat Nebula?%NEBULA_VER_CURRENT%/romdata/*.DAT >%NEBULA_VER_CURRENT%.tmp

REM --- Stage 1: Compare the previous and current drivers

mamediff -v %NEBULA_VER_PREVIOUS%.tmp %NEBULA_VER_CURRENT%.tmp
NOTEPAD mamediff.log
rm -f %NEBULA_VER_PREVIOUS%.tmp %NEBULA_VER_CURRENT%.tmp mamediff.log

This results in typical a MAMEDiff style output that shows the changes between the two versions. This only applies to 'romdata' games though, changes to the remaining games must be spotted in the release notes or within the emulator itself. Fortunately, Nebula only has about a dozen games that are not of the 'romdata' type though (compared to over 400 that are of the 'romdata' type).


Fixing Problems in the Nebula Drivers

It has already been mentioned that a few problems exist in the Nebula drivers themselves and need fixing. The problems that currently exist are 'CRC conflicts' which basically means that two ROMs in a parent and clone have the same name but a different CRC. This causes a problem when trying to merge ROMs using a ROM manager like RomCenter or ClrMamePro. Such conflicts can be identified by using the following batch script:

@ECHO OFF

SET NEBULA_EMU_VER=v2.24

REM --- Convert drivers into a valid dat and check datutil.log

cat Nebula?%NEBULA_EMU_VER%/romdata/*.dat Nebula?%NEBULA_EMU_VER%/romdata/*.DAT >tmp.dat
datutil -v -l tmp.dat
NOTEPAD datutil.log

rm -f tmp.dat datutil.dat datutil.log

The output will show problems that exist in the Nebula drivers. For example:

-------------------------------------------------------------------------------
Warning Summary
-------------------------------------------------------------------------------

ROM warnings:

    CRC Conflict

-------------------------------------------------------------------------------
Warning Details
-------------------------------------------------------------------------------

garoun - Garou: Mark of the Wolves (Decrypted C)
    ROM 253-v1.bin - CRC Conflict (263e388c)
    ROM 253-v2.bin - CRC Conflict (2c6bc7be)
    ROM 253-v3.bin - CRC Conflict (0425b27d)
    ROM 253-v4.bin - CRC Conflict (a54be8a9)

garoup - Garou: Mark of the Wolves (Prototype)
    ROM 253-v1.bin - CRC Conflict (274f3923)
    ROM 253-v2.bin - CRC Conflict (8f86dabe)
    ROM 253-v3.bin - CRC Conflict (05fd06cd)
    ROM 253-v4.bin - CRC Conflict (14984063)

garou - Garou: Mark of the Wolves
    ROM 253-v1.bin - CRC Conflict (263e388c)
    ROM 253-v2.bin - CRC Conflict (2c6bc7be)
    ROM 253-v3.bin - CRC Conflict (0425b27d)
    ROM 253-v4.bin - CRC Conflict (a54be8a9)

garouo - Garou: Mark of the Wolves (Set 2)
    ROM 253-v1.bin - CRC Conflict (263e388c)
    ROM 253-v2.bin - CRC Conflict (2c6bc7be)
    ROM 253-v3.bin - CRC Conflict (0425b27d)
    ROM 253-v4.bin - CRC Conflict (a54be8a9)

rbff2 - Real Bout Fatal Fury 2: The Newcomers
    ROM 240-p1.bin - CRC Conflict (b6969780)

rbff2a - Real Bout Fatal Fury 2: The Newcomers (Set 2)
    ROM 240-p1.bin - CRC Conflict (80e41205)

wh1 - World Heroes
    ROM 053-p1.bin - CRC Conflict (95b574cb)
    ROM 053-p2.bin - CRC Conflict (f198ed45)

wh1h - World Heroes (Alternate Set)
    ROM 053-p1.bin - CRC Conflict (ed29fde2)
    ROM 053-p2.bin - CRC Conflict (98f2b158)

Once the CRC conflicts have been identified, they need rectifying by re-naming the appropriate ROMs in the Nebula drivers. This will not affect Nebula itself since it loads ROMs by CRC (rather than loading them by name) but it will allow a good RomCenter/ClrMamePro data file to be created that matches the 'fixed' drivers.


Validation of the Data File for 'ROMDATA' Games

DatUtil enables you to convert Nebula drivers into a ClrMamePro or RomCenter data file and by using this facility, you can create a data file for use with MAMEDiff. The only thing to bear in mind is that Nebula drivers do not always list all of the ROMs in clones (where ROMs are available in the parent) so the data file generated from the 'romdata' could be missing some ROMs. This only seems to happen for CPS-2 games but to keep the size of the MAMEDiff report down, it is recommended to exclude all merged ROMs in the clones. This is done for both the data file being validated and the data file that is generated directly from the 'romdata' directory by using the Unix 'grep' command.

Comparison of the data file for 'romdata' games against the 'romdata' itself can therefore be done with the following batch script:

@ECHO OFF

SET NEBULA_EMU_VER=v2.24
SET NEBULA_DAT_VER=v2.24

REM --- Convert drivers into a valid dat, without merged ROMs

cat Nebula?%NEBULA_EMU_VER%/romdata/*.dat Nebula?%NEBULA_EMU_VER%/romdata/*.DAT >tmp.dat
datutil -l tmp.dat >nul
grep -v " merge " datutil.dat >driver.dat
rm -f tmp.dat datutil.dat datutil.log

REM --- Remove merged ROMs from the 'logiqx' dat

grep -v " merge " "Nebula %NEBULA_DAT_VER%.dat" >logiqx.dat

REM --- Compare the 'driver' dat against the 'logiqx' dat

mamediff -v driver.dat logiqx.dat
NOTEPAD mamediff.log
rm -f driver.dat logiqx.dat mamediff.log

If the data file for 'romdata' games is correct then MAMEDiff should report 'No differences found'.

You really need to understand what DatUtil and MAMEDiff do internally to know how and why the differences are picked up by this test. Basically though, the only errors that can go unnoticed are parent ROMs being listed in clones unnecessarily. This is because such ROMs would be treated as 'merged' ROMs and therefore excluded from the comparison (as a result of the ' grep' commands). However, they are fairly harmless because the ROM manager won't create them since they are merged/shared and will only be stored in the parent ZIP.


Validation of the Data File for Non-'ROMDATA' Games

To validate the data file for games without 'romdata' drivers, the best thing that can be done is to compare them against the RomCenter data file generated by Nebula itself. This relies on the accuracy of the RomCenter data file but it is the best that can be achieved before trying the games in the emulator itself. The following script uses MAMEDiff to show ROMs that are in the hand created data file but not the Nebula generated one.

@ECHO OFF

SET NEBULA_EMU_VER=v2.24
SET NEBULA_DAT_VER=v2.24

REM --- Convert RomCenter.dat names to lower case.

datutil -l -o "Nebula %NEBULA_EMU_VER%/NebulaCMPro.dat" "Nebula %NEBULA_EMU_VER%/NebulaRomCenter.dat" >nul

REM --- Compare the 'RomCenter' dat against the 'extra' dat

mamediff -d2 -r "Nebula %NEBULA_EMU_VER%/NebulaCMPro.dat" "Nebula %NEBULA_DAT_VER% extra.dat"
NOTEPAD mamediff.log
rm -f datutil.log mamediff.log mamediff.dat

What this comparison will show is ROMs in the hand created data file that are either incorrect (i.e. wrong name, size or CRC) and ROMs that are simply unneeded. Example output might be something like this:

Supplementary dat for use with Nebula v2.24 extra
Use in addition to a perfect split-merged set for NebulaCMPro

Created with the MAMEDiff option(s): -d2 -r 

Neo-Geo Universe BIOS v1.2 [name: uni-bios]

  ROM: uni-bios.rom ( 131072 bytes, crc 4fa698e9)

By looking at the Nebula generated data file, uni-bios.rom can be seen to have a CRC of 00000000 (so that any new versions can be used) so this is not an error in the hand created data file.

The only problems that can be missed by this check are ROMs that are completely missing from the hand created data file. Such problems must be identified by checking the games in the emulator itself, either by using the built-in ROM audit or by loading the game.


Combining the Two Data Files and Adding SHA1 Information

Once the two data files have been updated to match the latest version of the emulator, they can be joined using the 'cat' command:

cat "Nebula v2.24.dat" "Nebula v2.24 extra.dat" >full.dat

If an SHA1 information is missing, it can be added using a command something like this:

datutil -f listinfo -x -i "E:\ROMs\Nebula" full.dat

The resulting data file (called 'datutil.dat') can then be compared against the original data file and copied over the top if everything went OK.


Building ROMs and Checking them Within Nebula

Now that the data file has been completed, a ROM set can be created using CMPro/RomCenter and checked within the emulator.

With so many games, it is easiest to check each system one by one within the emulator (just look for non-green games). For example, checking PolyGameMaster games should show that they are all good and working:

 Nebula Screenshot

It is pretty unlikely that there will be a problem with the 'romdata' based games so pay particular attention to the systems that do not use 'romdata'.


Checking the Accuracy of the Nebula-Generated Data File

Once the hand-created data file has been tested, it can be used to check the accuracy of the data file generated by Nebula itself. There is a big problem with the Neo-Geo resource ROMs in the data file generated by Nebula so the best thing is to just exclude them from the comparison using the 'grep' command. A suitable batch script for doing the comparison might be as follows:

@ECHO OFF

SET NEBULA_EMU_VER=v2.24
SET NEBULA_DAT_VER=v2.24

REM --- Convert ElSemi data file into CMPro format, removing Neo-Geo ROMs and making names lower case

grep -v sfix.sfx "Nebula %NEBULA_EMU_VER%/NebulaRomCenter.dat"|grep -v neo-geo.rom|grep -v sm1.sm1|grep -v 000-lo.lo >elsemi.dat
datutil -l -o elsemi.dat elsemi.dat >nul

REM --- Remove Neo-Geo ROMs from the Logiqx data file

grep -v sfix.sfx "../Nebula %NEBULA_DAT_VER%.dat"|grep -v neo-geo.rom|grep -v sm1.sm1|grep -v 000-lo.lo >logiqx.dat

REM --- Compare the 'driver' dat against the 'logiqx' dat

mamediff -v elsemi.dat logiqx.dat
NOTEPAD mamediff.log
rm -f elsemi.dat logiqx.dat datutil.log mamediff.log

The output may look something like this and basically shows how the Nebula generated data file should be changed if it is to match the hand created data file:

-------------------- Games affected by set reorganisation --------------------

The King of Fighters '99: Millennium Battle (Non Encrypkof99 [name: kof99n - parent: kof99]
< rom ( name //kf99_s1d.rom size 131072 crc 1b0133fe )
> rom ( name kf99_s1d.rom size 131072 crc 1b0133fe sha1 7f8e743bc0c8384dec12ca7007d32b23fef8875d )

Sengoku 3 (Decrypted C) [name: sngoku3n - parent: sengoku3]
< rom ( name   size 0 flags nodump )

Super Street Fighter 2 Turbo (Asia 940223) [name: ssf2ta - parent: ssf2]
< rom ( name ssf.q05 merge ssf.q05 size 524288 crc 4770e7b7 )
< rom ( name ssf.q06 merge ssf.q06 size 524288 crc 4e79c951 )
< rom ( name ssf.q03 merge ssf.q03 size 524288 crc 695cc2ca )
< rom ( name ssf.q08 merge ssf.q08 size 524288 crc 6f5a088c )
< rom ( name ssf.q04 merge ssf.q04 size 524288 crc 9d9ebe32 )
< rom ( name ssf.q07 merge ssf.q07 size 524288 crc cdd14313 )

Super Street Fighter 2 Turbo (US 940223) [name: ssf2tur1 - parent: ssf2]
< rom ( name ssf.q05 merge ssf.q05 size 524288 crc 4770e7b7 )
< rom ( name ssf.q06 merge ssf.q06 size 524288 crc 4e79c951 )
< rom ( name ssf.q03 merge ssf.q03 size 524288 crc 695cc2ca )
< rom ( name ssf.q08 merge ssf.q08 size 524288 crc 6f5a088c )
< rom ( name ssf.q04 merge ssf.q04 size 524288 crc 9d9ebe32 )
< rom ( name ssf.q07 merge ssf.q07 size 524288 crc cdd14313 )

Super Street Fighter 2 X (Jap 940223) [name: ssf2xj - parent: ssf2]
< rom ( name ssf.q05 merge ssf.q05 size 524288 crc 4770e7b7 )
< rom ( name ssf.q06 merge ssf.q06 size 524288 crc 4e79c951 )
< rom ( name ssf.q03 merge ssf.q03 size 524288 crc 695cc2ca )
< rom ( name ssf.q08 merge ssf.q08 size 524288 crc 6f5a088c )
< rom ( name ssf.q04 merge ssf.q04 size 524288 crc 9d9ebe32 )
< rom ( name ssf.q07 merge ssf.q07 size 524288 crc cdd14313 )

< NeoGeo Universe BIOS [name: uni-bios]
> Neo-Geo [name: neogeo]
< rom ( name uni-bios.rom size 131072 flags nodump )

---------------------- Games requiring additional ROMs -----------------------

Puzzle Bobble  [name: pbobblen]
> rom ( name 083-v3.bin size 1048576 crc 0840cbc4 sha1 1adbd7aef44fa80832f63dfb8efdf69fd7256a57 )
> rom ( name 083-v4.bin size 524288 crc 0a548948 sha1 e1e4afd17811cb60401c14fbcf0465035165f4fb )

Puzzle Bobble (Set 2) [name: pbobblna - parent: pbobblen]
> rom ( name 083-v3.bin merge 083-v3.bin size 1048576 crc 0840cbc4 sha1 1adbd7aef44fa80832f63dfb8efdf69fd7256a57 )
> rom ( name 083-v4.bin merge 083-v4.bin size 524288 crc 0a548948 sha1 e1e4afd17811cb60401c14fbcf0465035165f4fb )

Super Street Fighter 2 Turbo (ETC 940223) [name: ssf2t - parent: ssf2]
> rom ( name sfx.02 size 131072 crc 0022633f sha1 cab3afc79da53e3887eb1ccd1f4d19790728e6cd )
> rom ( name sfx.09 size 524288 crc 642fae3f sha1 746df99b826b9837bba267104132161153c1daff )
> rom ( name sfxe.08 size 524288 crc 74c24062 sha1 f3eca09e0544c6aa46b0c4bead2246ab1e9a97d9 )
> rom ( name sfxex.08 size 524288 crc d7436ae9 sha1 1861f9c81ff2ffd956b94412ef6eb096f7558b96 )

Super Street Fighter 2 Turbo (Asia 940223) [name: ssf2ta - parent: ssf2]
> rom ( name sfx.02 size 131072 crc 0022633f sha1 cab3afc79da53e3887eb1ccd1f4d19790728e6cd )
> rom ( name ssf.19 merge ssf.19 size 2097152 crc 34e825c5 sha1 4d320fc96d1ef0b9928a8ce801734245a4c097a5 ) *already in MAME
> rom ( name sfxa.08 size 524288 crc 39be596c sha1 f7ab80e64cbb703535dd39b875273eefa57df489 )
> rom ( name ssf.15 merge ssf.15 size 2097152 crc 5eb703af sha1 4b302dbb66e8a5c2ad92798699391e981bada427 ) *already in MAME
> rom ( name sfx.09 size 524288 crc 642fae3f sha1 746df99b826b9837bba267104132161153c1daff )
> rom ( name sfxax.08 size 524288 crc 7c5fd202 sha1 e08db90a71528b8479de820cd6d6a7304de9775c )
> rom ( name ssf.13 merge ssf.13 size 2097152 crc cf94d275 sha1 bf2a6d98a656d1cb5734da7836686242d3211137 ) *already in MAME
> rom ( name ssf.17 merge ssf.17 size 2097152 crc ffa60e0f sha1 b21b1c749a8241440879bf8e7cb33968ccef97e5 ) *already in MAME

Super Street Fighter 2 Turbo (US 940223) [name: ssf2tur1 - parent: ssf2]
> rom ( name sfx.02 size 131072 crc 0022633f sha1 cab3afc79da53e3887eb1ccd1f4d19790728e6cd )
> rom ( name ssf.19 merge ssf.19 size 2097152 crc 34e825c5 sha1 4d320fc96d1ef0b9928a8ce801734245a4c097a5 ) *already in MAME
> rom ( name ssf.15 merge ssf.15 size 2097152 crc 5eb703af sha1 4b302dbb66e8a5c2ad92798699391e981bada427 ) *already in MAME
> rom ( name sfx.09 size 524288 crc 642fae3f sha1 746df99b826b9837bba267104132161153c1daff )
> rom ( name sfxu.08 size 524288 crc b3c71810 sha1 b51515f4f4aee5bbbfc8b79372d0bc6e0c140912 )
> rom ( name ssf.13 merge ssf.13 size 2097152 crc cf94d275 sha1 bf2a6d98a656d1cb5734da7836686242d3211137 ) *already in MAME
> rom ( name sfxux.08 size 524288 crc fa2396a6 sha1 80da9a5355eebd9dc2f3846020338966a1de596f )
> rom ( name ssf.17 merge ssf.17 size 2097152 crc ffa60e0f sha1 b21b1c749a8241440879bf8e7cb33968ccef97e5 ) *already in MAME

Super Street Fighter 2 X (Jap 940223) [name: ssf2xj - parent: ssf2]
> rom ( name sfx.02 size 131072 crc 0022633f sha1 cab3afc79da53e3887eb1ccd1f4d19790728e6cd )
> rom ( name sfxj.08 size 524288 crc 2de76f10 sha1 8cbe96dfeaa41306caa2819b82272ce3b0b9f926 )
> rom ( name ssf.19 merge ssf.19 size 2097152 crc 34e825c5 sha1 4d320fc96d1ef0b9928a8ce801734245a4c097a5 ) *already in MAME
> rom ( name ssf.15 merge ssf.15 size 2097152 crc 5eb703af sha1 4b302dbb66e8a5c2ad92798699391e981bada427 ) *already in MAME
> rom ( name sfx.09 size 524288 crc 642fae3f sha1 746df99b826b9837bba267104132161153c1daff )
> rom ( name sfxjx.08 size 524288 crc 656a9858 sha1 58e2364a945cc0a52079cb170e72abd436660610 )
> rom ( name ssf.13 merge ssf.13 size 2097152 crc cf94d275 sha1 bf2a6d98a656d1cb5734da7836686242d3211137 ) *already in MAME
> rom ( name ssf.17 merge ssf.17 size 2097152 crc ffa60e0f sha1 b21b1c749a8241440879bf8e7cb33968ccef97e5 ) *already in MAME

------------------------------- Game additions -------------------------------

> Neo-Geo Universe BIOS v1.2 [name: uni-bios]
> Vampire Hunter 2: Darkstalkers Revenge (Japan 970913) [name: vhunt2r1]

As can be seen above, there are a variety of unneeded ROMs, missing ROMs, missing games and peculiarities in the Nebula generated data file. Brief explanations are as follows:

  • Incorrect ROMs in clones of ssf2 (ssf2t, ssf2ta, ssf2tur1 and ssf2xj).
  • Missing sound ROMs in pbobblen and pbobblna.
  • The game vhuntr1 is missing (perhaps because the driver refers to a non-existent parent called vhunt).
  • A badly named ROM in kof99n (it looks like there has been an attempt to comment it out).
  • A strange ROM entry in sngoku3n.

Note: The uni-bios difference is due to the fact that Nebula does not specify a CRC for uni-bios.rom but the hand-created data file does. It is therefore not an error in either data file.