|
Last updated: 30th June 2004 IntroductionDats are required by ROM managers such as CMPro and RomCenter so that they can build and audit ROM sets for use in emulators. It is important to ensure that your dats are correct otherwise the ROM manager cannot do its job properly and may actually damage ROM sets. This document and the other supplementary documents should contain most of the details you need to know about data file creation and the potential pitfalls. At this time there are two main formats and if you do decide to create a data file I would urge you to release it in both formats. Converting from one format to another is very easy using DatUtil so there is really no excuse for not doing so! General GuidelinesStandardsMAME uses lower case filenames for arcade ROMs so if you are creating a data file for a new arcade emulator I would recommend you stick to this convention (whether or not the ROMs you have use lower case filenames). Lower case filenames are much easier on the human eye also. CRCs should be 8 characters long (use leading zeros if necessary), they should be lower case and not use the prefix '0x'. All of these points are for the sake of consistency and readability. ClrMamePro and my tools can both load CRCs with a prefix of '0x' but I think it looks ugly! MergingMany emulators support ROM merging in order to save on the amount of disk space used. What this means is that ROMs used by more than one game need only be stored once. When games can be merged, one of them is called the 'parent' and the others are called the 'clones'. You cannot have a clone of a clone, only one parent with one or more clones. e.g. In MAME there is a game called '1942' which is a parent and '1942a/1942b' are clones. There are three types of ROM sets:
If an emulator supports ROM merging then the data files you produce should specify the ROM merging correctly. If you do not specify merging correctly then it not only prevents people from choosing the merge option they would like but can also cause misleading scan results. Take PacMan and one of its clones as an example: game ( name pacman description "PuckMan (Japan set 1)" year 1980 manufacturer "Namco" rom ( name namcopac.6e size 4096 crc fee263b3 ) rom ( name namcopac.6f size 4096 crc 39d1fc83 ) rom ( name namcopac.6h size 4096 crc 02083b03 ) rom ( name namcopac.6j size 4096 crc 7a36fe55 ) rom ( name pacman.5e size 4096 crc 0c944964 ) rom ( name pacman.5f size 4096 crc 958fedf9 ) ) game ( name npacmod description "PuckMan (harder?)" year 1981 manufacturer "Namco" cloneof pacman romof pacman rom ( name namcopac.6e merge namcopac.6e size 4096 crc fee263b3 ) rom ( name namcopac.6f merge namcopac.6f size 4096 crc 39d1fc83 ) rom ( name namcopac.6h merge namcopac.6h size 4096 crc 02083b03 ) rom ( name npacmod.6j size 4096 crc 7d98d5f5 ) rom ( name pacman.5e merge pacman.5e size 4096 crc 0c944964 ) rom ( name pacman.5f merge pacman.5f size 4096 crc 958fedf9 ) ) It can be seen that npacmod only has one ROM that differs from the parent set so some people may be tempted to create a data file that forces split sets by doing this: game ( name pacman description "PuckMan (Japan set 1)" year 1980 manufacturer "Namco" rom ( name namcopac.6e size 4096 crc fee263b3 ) rom ( name namcopac.6f size 4096 crc 39d1fc83 ) rom ( name namcopac.6h size 4096 crc 02083b03 ) rom ( name namcopac.6j size 4096 crc 7a36fe55 ) rom ( name pacman.5e size 4096 crc 0c944964 ) rom ( name pacman.5f size 4096 crc 958fedf9 ) ) game ( name npacmod description "PuckMan (harder?)" year 1981 manufacturer "Namco" rom ( name npacmod.6j size 4096 crc 7d98d5f5 ) ) Even though this would work to a degree it is a bad idea for two reasons:
Similarly, people may be tempted to create a fully merged data file as follows: game ( name pacman description "PuckMan (Japan set 1) and PuckMan (harder?)" year 1980 manufacturer "Namco" rom ( name namcopac.6e size 4096 crc fee263b3 ) rom ( name namcopac.6f size 4096 crc 39d1fc83 ) rom ( name namcopac.6h size 4096 crc 02083b03 ) rom ( name namcopac.6j size 4096 crc 7a36fe55 ) rom ( name pacman.5e size 4096 crc 0c944964 ) rom ( name pacman.5f size 4096 crc 958fedf9 ) rom ( name npacmod.6j size 4096 crc 7d98d5f5 ) ) This too would be incorrect for three reasons:
If an emulator only supports one of the three merging options then you can specify it in a CMPro data file using the 'forcemerging' option in the header. Do not create a data file containing incorrect merging information. It is a terrible sin! The syntax for specify merging information is detailed in the CMPro and RomCenterformat documents. Special CRCs and FlagsThere are some special CRCs used by ROM managers and at times you may find them useful:
Important note: 00000000 CRCs should not be used in RomCenter datafiles to list non-dumped games. If a game hasn't been dumped then you can neither know the ROM names or their sizes. So, if a game hasn't been dumped yet then don't put it in a data file (after all, a data file is to identify and validate ROMs). If you want to document non-dumped games then provide a little text file with your data file! DisksIf an emulator uses disk images then you can also include this information in a CMPro data file. At this time it is not possible to put this information into RomCenter. SamplesIf an emulator uses sound samples then you can also include this information in a CMPro data file. At this time it is not possible to put this information into RomCenter so a dedicated 'samples' data file is a sensible option if the samples are not packaged with the emulator itself. Creating a Data FileUse CMPro as Your Master Format!I would seriously recommend that you do all of your data file creation in CMPro format and convert to RomCenter afterwards. The main reason for this is because is is easier to read, less prone to errors during editing and you will spot more mistakes too. Once a data file has been loaded into CMPro and is known to be free of errors it will convert to RomCenter format without problems. There is also a lot of additional information that can be stored in the CMPro data file format and you could not maintain it if RomCenter was your master format. Scanning Directories/ZIPsThe quickest way to create a data file from a set of ZIP files is to scan the ZIPs using a data file creation tool. The best approach nowadays is probably to use DatUtil:
You should be cautious when scanning directories or ZIPs because you could end up including files that do not belong in the data file (readme files, jed files, etc.). The filenames could also be a mixture of upper and lower case whereas it has already been mentioned that use of lowercase is preferable. The mixed case can be fixed with the '-l' option of DatUtil though. Creating by Hand (i.e. Text Editor)When creating a data file for an emulator where the games are already emulated by MAME (or any other emulator for that matter), it is generally a good idea to copy and paste the ROM details from the existing emulator's data file. This can either be done manually (copy and paste) or using the '-g' option of DatUtil. When maintaining data files for the CPS-2 emulators I tend to copy and paste games directly from the CPS-2 data file. After using the data file in CMPro/RomCenter I then check that the game works correctly in the emulator. Use Data File HeadersBoth the CMPro and RomCenter data files have headers to specify details such as the emulator name, version number, data file author, etc. Both emulators use this information so you should fill it in correctly. An explanation of the various fields is within the CMPro and RomCenter sections of this document. The header file of a CMPro data file can also specify the type ROM merging required by the emulator (non-merged, split-merged or fully-merged). RomCenter has a similar option since the introduction of the 2.50 format but it does not currently have any effect after the data file is first loaded. Include Merge InformationROM merging has been discussed earlier in this document so make sure you do it correctly! Converting Between FormatsTo convert data files from one format to another use DatUtil. This will convert all of the game information and header information for you and if your data files are correct there will be no problems at all with this procedure. When attempting to convert some RomCenter data files to CMPro format there are two common faults that can cause problems:
To get around these problems you must fix the old RomCenter file by hand. TestingChecking Data Files for ErrorsThere are a variety of methods you can use to test your data file for syntactical, logical and lexical errors.
Use MAMEDiff to Check Against the Emulator!If the emulator can generate its own data file or the emulator is natively supported by MAMEDiff then make use of this fact. For example:
If the emulator can't generate its own data file then you may have to resort to a hex editor if you want to check ROM names! Before using the hex editor, make sure that you decompress the executable if it has been packed with UPX (use the '-d' option of UPX). Build a ROM SetCreate a complete ROM set using CMPro then scan it with CMPro and/or RomCenter. Scan a ROM SetTry scanning what you think is a clean ROM set using both CMPro and RomCenter. Both should scan the set perfectly and not produce contradictory results. Do not just scan with RomCenter as it is not as strict as CMPro and you may miss some errors in your data file as a result. Checking a ROM Set from the EmulatorSome emulators will include a ROM audit feature. Run this over the set that was produced by your data file. If there is no audit feature then the only way to be sure the sets are correct is to load every game! When using an emulator to check sets I would highly recommend that you use split-merged sets (assuming the emulator supports them). If a split-merged set works without problems then you can be sure that a fully-merged or non-merged set will also work (the reverse is not true however). Checking MAME CompatibilityThis is not really necessary but can be of interest. If you wish to know whether an emulator is 100% MAME compatible then do these two tests using CMPro:
Note: If either scan shows any other kind of errors (i.e. wrong filename for a ROM), then there is probably some incompatibility. Problems EncounteredNow and then you may encounter something that is difficult to represent accurately in a ROM manager data file. Missing ParentsSometimes an emulator will support a clone (or clones) but not the parent game itself. If the emulator searches a parent directory/ZIP when loading ROMs then it may be advantageous to include the parent game in the data file as well (rather than make the clone(s) standalone games with no parent defined). The ROM manager will report more games than are actually supported by the emulator but it does have the big advantage that a MAME-compatible emulator can have a MAME compatible data file (whereas removing the parent/clone relationship would prevent this). This is common in the data files for the CPS-2 emulators because parent games are not always decrypted before the clones. Impossible Merging/CRC ConflictsOccasionally an emulator may use the same filename for two ROMS in a parent and clone where the ROMs themselves differ (i.e. same name but different CRC). This would obviously cause a problem within a ROM manager because it is physically impossible to create a fully merged ROM set. The only way to solve this is to remove the parent/clone relationship until such time as the emulator is changed to use different ROM names. Alternatively, if the emulator can load ROMs by their CRC then you could rename the ROMs in your data file. Note: CMPro and RomCenter will report an error in any data file that contains conflicting CRCs in parents and clones. |