DatUtil was created to aid in the creation of dat files for Rom Managers such as ClrMamePro and RomCenter (Roman Scherzer / Eric Bole-Feysot). It can convert between formats, extract individual games (and optionally their clones) and cleanse dats. It supports a variety of formats...

Load and Save:

- MAME ListInfo (ClrMamePro, MAME, MESS, RAINE, Shark)
- RomCenter 2.50
- MAME ListXML (MAME and MESS)
- Generic XML
- Tab Delimited
- AAE RomList

Load only:

- RomCenter 1.90
- RomCenter 2.00
- Nebula Drivers
- Calice Drivers
- Zinc --list-sets
- GNGEO romrc
- ClrMame Log
- Folder Scan (subdirectories and zips)

Save only:

- MAME GameList
- Title List (for listing game titles - used by CPS-2 readme and RAINE docs)
- RomCenter Sublist
- Game Launcher / GameEx Map File
- Hyperspin Database

13 April 2009 - DatUtil v2.46

- Uses DatLib v2.36.
- Added support for 64-bit ROM sizes (n.b. not for directory scan).

9 January 2009 - DatUtil v2.45

- Uses DatLib v2.35.
- Fixed warning messages for ROM size conflicts (Kawaks v1.60).

2 November 2008 - DatUtil v2.44

- Uses DatLib v2.34.
- Changed the order of MD5 and SHA1 in listinfo and XML datafiles.
- Removed the ambiguous "-f xml" option. Use listxml or genericxml.

28 October 2008 - DatUtil v2.43

- Uses DatLib v2.33.
- Added SHA1 / MD5 information for Neil Corlett's KGPE hack.
- Added load / support for AAE RomList.
- datutil -k now allows MD5 and SHA1 at the same time.
- datutil -n will NOT determine file types (everything = roms).
- Changed the directory scanner to process zero byte files / roms.
- Support for a new rom / disk status of 'verified'
- Added save support for the Hyperspin Database format.

16 July 2008 - DatUtil v2.42

- Uses DatLib v2.32.
- Improved datutil -i to search for ROMs by CRC as well as name.

29 June 2008 - DatUtil v2.41

- Uses DatLib v2.31.
- Added support for ' in the XML formats.

24 June 2008 - DatUtil v2.40

- Uses DatLib v2.30.
- Added XML entity support to generic XML headers (when saving).

10 June 2008 - DatUtil v2.39

- Uses DatLib v2.29.
- Fixed parsing of listinfo comments.
- Added warning when forcenodump is missing.

16 April 2008 - DatUtil v2.38

- Uses DatLib v2.27.
- Fixed generation of url within CMPro datafiles (needs quotes).
- Added support for release information (mainly for console ROMs).
- Added release support to list+generic xml, listinfo, delimited.
- Added the <release> element to the ROM Management Datafile DTD.
- Added support for Game Launcher and GameEx Map Files (untested).
- Added support for substring selections (within descriptions).
- Added the -S option for selections based on game descriptions.

6 April 2008 - DatUtil v2.37

- Uses DatLib v2.26.
- Minor tweaks to the DTD (header and manufacturer now optional).
- Fixed empty header being written to Generic XML datafiles.
- Fixed conversion of MESS XML to Generic XML (machines -> games).

4 April 2008 - DatUtil v2.36

- Uses DatLib v2.25.
- Added automatic date population (based on the version number).
- Renamed forcezipping to forcepacking (CMPro).
- Renamed forcedrommode to rommode (RomCenter 3).
- Renamed forcedbiosmode to biosmode (RomCenter 3).
- Renamed forcedsamplemode to samplemode (RomCenter 3).
- Changed from m/s/n to merged/split/unmerged (RomCenter 3).
- Changed from y/n to yes/no (RomCenter 3).
- Added fix summary for header items (appears in datutil.log).
- Added warnings for invalid header items (appear in datutil.log).
- Added notices for lost header items (appear in datutil.log).
- Changed -Z to -P (although -Z also works as before).

24 March 2008 - DatUtil v2.35

- Uses DatLib v2.24.
- Fixed crash (allzip had spurious carriage returns in kyukaidk).

17 March 2008 - DatUtil v2.34

- Uses DatLib v2.23.
- Added date, email, homepage, url, comment, forcenodump to CMPro.
- Added category to RomCenter header (for completeness).
- Added the Generic XML format.
- Note: CMPro, RomCenter and XML headers all contain the same info.
- Added -T, -E, -H, -U, -O and -N (options for header information).

26 November 2007 - DatUtil v2.33

- Uses DatLib v2.22.
- Fixed crash when loading full MESS XML (0.105 upwards).
- MESS: Support for new device attributes (type, tag and mandatory).
- MESS: Support for machine "RAM options".
- MAME + MESS: Support for new emulator "debug" attribute.
- MAME + MESS: Changed "refresh" attributes from float to double.
- MAME + MESS: Added driver "savestate" warnings.

29 July 2007 - DatUtil v2.32

- Uses DatLib v2.21.
- Empty attributes are now ignored (e.g. sha1="")
- Removed cloneof, romof and sampleof fixes from datutil -i
- DatUtil -i, -I and -p now default the output format to CMPro (in addition to -g, -G and -r which already behave like that)
- Added support for the isbios="yes" of MAME v0.117u2 onwards.

1 January 2007 - DatUtil v2.31

- Uses DatLib v2.20.
- Added 'prune' option (-p) for removing roms, disk and/or samples
- Added 'incorporate' option (-I) to assist in combining datafiles

17 October 2006 - DatUtil v2.30

- Uses DatLib v2.18.
- More intelligent DTD creation when saving in ListXML format
- Added support for game/display (MAME v0.107)
- Added support for game/input/control (MAME v0.107)
- Added support for emulator/build (MAME+MESS v0.101u4 upwards)
- Added support for driver/savestate (MAME v0.101u1 upwards)
- Note: The above changes affect the tab delimited format! - Added 'delimited.sql' as an example of loading dats into MySQL

16 July 2006 - DatUtil v2.29

- Uses DatLib v2.17.
- Fixed bug that caused the DatUtil -g and -r options to crash

11 July 2006 - DatUtil v2.28

- Uses DatLib v2.16.
- Fixed bug that messed up internal indices (affected crc matching)
- More intelligent directory scanner (regarding samples and CHDs)
- Removed 64MB file limit from directory scanner (reads 1MB blocks)

13 January 2006 - DatUtil v2.27

- Uses DatLib v2.15.
- Last release would not load data files less than 4KB in size!

12 January 2006 - DatUtil v2.26

- Uses DatLib v2.14.
- Added a save-only format to imitate MAME's gamelist.txt file.
- File identification is now done by loading just the first 4KB.
- Added a list of supported save-formats to the command line help.

6 December 2005 - DatUtil v2.25

- Uses DatLib v2.13.
- DatLib v2.12 broke the RomCenter drivers (now fixed).

5 December 2005 - DatUtil v2.24

- Uses DatLib v2.12.
- Fixed a rare crash (if the longest line in the dat was the last).
- Fixed a problem loading RomCenter data files containing [].

13 November 2005 - DatUtil v2.23

- Uses DatLib v2.11.
- Added support for 'board', 'colordeep', 'credits' and 'archive'
(fully supported by ListInfo, ListXML and Delimited formats).
- M1 XML loader now retains the 'board' information (see above).
- M1 XML will not use hex for ROM sizes after all... changed back!
- Improved the way that the Nebula driver handles Neo-Geo ROMs.
- Fixed Nebula Jukebox driver for Neo-Geo ROMs (broken in v2.7).

31 October 2005 - DatUtil v2.22

- Uses DatLib v2.10.
- Fixed identification of ListXML for MAME+MESS v0.101u4 onwards.
- Added the '-q' option (to save Listinfo with quotes throughout).

28 October 2005 - DatUtil v2.21

- Uses DatLib v2.9.
- M1 XML will use hex for ROM sizes. Changed driver accordingly.
- Now reports duplicate game descriptions and CRCs (i.e. ROMs).

5 October 2005 - DatUtil v2.20

- Uses DatLib v2.8.
- Improved the logerror() function (used for reporting ZIP errors).
- Added support for M1's XML format (when it becomes available).

17 September 2005 - DatUtil v2.19

- Uses DatLib v2.7.
- Compiled with zlib 1.3.
- Improved Neo-Geo handling for the Nebula driver.
- Improved 'romof' handling for RomCenter 2 driver (during loading).

31 August 2005 - DatUtil v2.18

- Uses DatLib v2.6.
- Fixed a 'hang' that occurred with circular cloneof relationships.

10 April 2005 - DatUtil v2.17

- Uses DatLib v2.5.
- Excludes disks from RomCenter data files (-k will include them).

25 March 2005 - DatUtil v2.16

- Uses DatLib v2.4.
- Fixed another crash with datutil -X and 'forcemerging full'.
- Changed the delimited format to use \N for NULLs.
- Changed the delimited format so that all types are 'game_*'.
- Changed the delimited format so that games include 'runnable'.
- Added save support for a DISK section in the RomCenter format.
- Added several aliases for the output formats (see notes above).

7 March 2005 - DatUtil v2.15

- Uses DatLib v2.3.
- Fixed a problem with counting parents (happened with DatUtil -X).
- This also avoids a crash with datutil -X and 'forcemerging full'.

28 February 2005 - DatUtil v2.14

- Uses DatLib v2.2.
- Added support for tab delimited files ('-f delimited').
- Added save-support for RomCenter sublists ('-f sublist').
- Added \t support to the listinfo driver (and tab delimited).
- Fixed problem of unrecognised escape sequences looping forever!

22 February 2005 - DatUtil v2.13

- Uses DatLib v2.1.
- Added a test option (-t) where nothing is actually saved.
- Log file is now automatically deleted if it is empty.
- Made selections more flexible (can now use commas as a delimiter).
- Added warnings for invalid game/sourcefile selections (-g and -G).
- Added warning messages for duplicated games (i.e. same game name).
- Can identify listinfo files even if they only contain resources.
- The 'remove clones' option will retain the original dat format.
- Fixed a crash caused by empty selection files (the '@' syntax).
- Saving a RomCenter datafile will now report an error if the delimiter
symbol is used in the game or ROM details (i.e. the dat is bad).
- Changed the error messages that are given when files are missing.
- Changed the command line help a little bit (explanation of -!).

15 February 2005 - DatUtil v2.12

- Uses DatLib v2.0.
- Compiled with GCC 3.4.2 (previously used GCC 3.2.3).
- Rewrote the manual, please read it from start to end. ;)
- Changed the summary of switches produced by DatUtil -?.
- Added the -k option to keep all details (chips, dips, etc).
- Added the -G option to select games from a specified sourcefile.
- Added the -! option so that -g and -G can be used as exclusions.
- Added the -X option to switch off 'fix merging' (use with -g).
- Added the -D option to switch off 'remove duplicates' (as above).
- Removed the -z and -y options as they are redundant now.

- DatLib v2.0 (enhancements relating to DatUtil):
- Added 'sourcefile' support to the listinfo and listxml drivers.
- Supports all known details in listinfo and listxml (-k option).
- N.b. ROM region is not loaded unless using the datutil -k option.
- Supports 'history' entries (when using the -k option).
- Added support for \\, \n, \xXX and \" in ListInfo (e.g. history).
- Improved entity handling (e.g. &amp; &#X;) in the ListXML parser.
- Supports 2 additional MESS objects (devices and extensions).
- DatLib can now save data in listxml format (MAME and MESS).
- Complemented CRCs are reported (should use flags nodump instead).
- Added checks for MD5 and SHA1 conflicts (CRCs already checked).
- Added support for external selection files with the @ symbol.
- Removed the hack made in DatLib v1.13 for Calice drivers.
- Removed funny size and missing year checks (pretty useless now).

16 January 2005 - DatUtil v2.11

- Uses DatLib v1.14.
- Added support for the GNGEO romrc file.
- Increased maximum ROM size to 64MB (used when scanning folders).

4 January 2005 - DatUtil v2.10

- Uses DatLib v1.13.
- Added support for Calice drivers.

25 November 2004 - DatUtil v2.9

- Uses DatLib v1.12.
- Improved the handling of CRC conflicts within the 'fix merging'.

26 October 2004 - DatUtil v2.8

- Uses DatLib v1.11.
- Added a parser for the ZiNc --list-sets.

18 August 2004 - DatUtil v2.7

- Uses DatLib v1.10.
- Better handling of 'clones of clones' when loading data files.

30 July 2004 - DatUtil v2.6

- Uses DatLib v1.9.
- Added full support for disk 'merge' and 'flags' information.

22 July 2004 - DatUtil v2.5

- Uses DatLib v1.8.
- Removed the need for xml2info by adding my own MAME XML parser.
- A nice side effect is that names like "Alien³: The Gun" work now!

21 July 2004 - DatUtil v2.4

- Uses DatLib v1.7
- Added support for MESS XML.
- Directory scanner now supports CHDs (when in a game directory).
- Improved 'fix merging' (where clone ROMs are not in the parent).
- Enhanced 'fix merging' to work for disks (SHA1/MD5 are copied).
- Added check for SHA1/MD5 conflicts of disks across clones.

17 July 2004 - DatUtil v2.3

- Uses DatLib v1.5
- Added the -j option for Nebula Jukebox (Z80 and Samples only).

11 July 2004 - DatUtil v2.2

- Uses DatLib v1.4:
- Added support for comments between games in CMPro data files.
- Added an option to ignore missing year information (-y).
- Added an option to select MD5 usage rather than SHA1 (-m).
Note that '-m' was previously used for the 'fix merging' option!
- Added an option to include missing info from another dat (-i).
Games: cloneof, romof, sampleof (good after using the -g option).
ROMs: size, crc, md5, sha1, flags (good for adding SHA1, etc).

4 July 2004 - DatUtil v2.1

- Added a message to say that the -m option is not required now.
- Uses DatLib v1.1:
- ClrMamePro/RomCenter header is now ignored when using OPTION_GAME.
- Removed a warning when you try using 'datutil -g <resource> -c'.
- Added debug information to identify spurious warnings like above.
- Added the -x option to calculate SHA1/MD5 when scanning ZIPs.
- n.b. SHA1/MD5 used to be calculated by default in DatUtil v2.0.

2 July 2004 - DatUtil v2.0

Wow, I never thought I'd get around to this rewrite but here it is!

Uses DatLib - The new library for handling dats of many formats

- A generic dat structure with various loading and cleansing routines.

- Far better memory handling. No longer uses huge blocks of fixed size memory.

Info: DatUtil used to allocate almost 45MB for a dat when it was loaded.
This was without MD5/SHA1 support and for a limited number of games!
DatLib now allocates only what it really requires (which is usually
about twice the size of the dat file).

- Multi-phase processing (pre-parse, parse, cleanse, save):

- Pre-parser translates dat into an intermediate dat format.
- Parser interprets the intermediate format to populate the dat structure.
- Cleanser does things like 'lowercase', 'remove dups', 'fix merging', etc.
- Saving outputs the cleansed dat structure in the required output format.

Note: The main difference is the addition of the pre-parser (which removes
common operations from the different dat parsers) and the separation
of the cleansing operations from the loader (they used to be embedded
in the various parsers, particularly painful for RomCenter format!).

- No limit on the number of games, ROMs, disks, samples etc.

- No limit on the length of file names, game titles, manufacturer names etc.

- Improved size and CRC handling (now numeric, not text based).

- Supports 'rebuildto' information for games.

- Supports alternative checksums (MD5 and SHA1).
N.B. The directory scanner also generates MD5 and SHA1 checksums for files.

- Supports 'region' information of ROMs.

- Improved driver system for developers adding new dat formats.

- Fully supports the latest RomCenter dat format (2.50).

- Improved RomCenter parser is not case sensitive like the old one was.

- All cleansing features can now be used with all supported formats.

- Improved 'fix merging'. Now operates across clones, not just parent/clone.
Also fixes bad parent references and prevents clones of clones.
The feature is run by default when a data file is loaded into memory.
A summary of fixes is shown in datutil.log, use -v option for full details.

- Duplicate ROMs are now removed when the data file is loaded into memory.

- Added debugging information (to help me track down problems).

- Removed alternate ROM support. It was only used by Shark and made data file
comparisons unnecessarily complicated!

- Removed code that fixed the bad parent 'pc' in MESS dats (the default
'fix merging' code does the same job now but without being hard coded).

- ClrMamePro log parser now supports disk images and alternative checksums.

- Nebula parser is now much better. With the default 'fix merging' behaviour,
the data file that is generated should be almost usable now!

- Improved baddump and nodump handling.
Also supports the syntax of MAME v0.68 (where the word 'flags' was missing).

- Improved logging. Added a verbose (-v) option for even more detail!

- You are given a warning if you try to convert a RomCenter data file where
the plugin is not 'arcade.dll'. It is likely that such a conversion will
not work correctly in ClrMamePro.

- The -g and -c options can now be used on resources. You can therefore
extract games for neogeo, zn1, etc. (e.g. 'datutil -g cpzn1 -c mame.dat').

- Removed the dat comparison facility. Use MAMEDiff instead!

- Added load support for MAME ListXML format (uses the xml2info tool).

- Added a warning summary to datutil.log.
Warnings are for missing details, funny ROM sizes and CRC conflicts.
For full details of the warnings, use the DatUtil -v (verbose) option.

- Added details of all information that is lost when saving in a different
format (e.g. Years and Manufacturers are lost when using RomCenter format).

Unsupported formats - Not included in the rewrite as they are of little use!

Load and Save:

- Calice DRV (drivers)

Load only:

- Callus gamelist
- CPSE data (drivers)
- Mimic HWC (drivers)
- System16 GCS (drivers)
- Sparcade RAT (drivers)
- Retrocade Romlist
- JAS romlist
- JFF text

26 February 2004 - DatUtil v1.32

- Finally added disk support for MAME!

31 January 2004 - DatUtil v1.31

- When saving data files in listinfo format, 'crc 00000000' is now
written out as 'flags nodump'.

24 January 2004 - DatUtil v1.30

- Increased the maximum number of games from 5000 to 6000.
- Removed support for the old RomCenter 1 format...
The old format is inadequate for modern day MAME releases.
- Removed knowledge of the MAME resource names. RomCenter 2 data
files must therefore have a [RESOURCES] section from now on.

20 January 2004 - DatUtil v1.29

- Minor adjustment to the routine that decides if a ROM lives in a resource.
This fixes the handling of shootgal/sg-01-0.

31 December 2003 - DatUtil v1.28

- Added knowledge of the atpsx resource.

29 December 2003 - DatUtil v1.27

- Added knowledge of the two new resources added in MAME v0.78.

14 September 2003 - DatUtil v1.26

- Changed names of Neo-Geo BIOS ROMs in the Nebula driver reader.

25 August 2003 - DatUtil v1.25

- Added knowledge of the seven new resources added in MAME v0.72.

19 May 2003 - DatUtil v1.24

- Added support for '[RESOURCES]' in RomCenter2 data files.

19 May 2003 - DatUtil v1.23

- Added a fix for the 'nodump' syntax of MAME v0.68.

7 April 2003 - DatUtil v1.22

- Fixed CMPro log scanner to support the syntax 'bad dump'.
- Added knowledge of konamigx resource.

18 January 2003 - DatUtil v1.21

- Removed 'cloneof neogeo' from 'neocd' in Nebula driver.
- Added knowledge of skns and stvbios resources.

20 May 2002 - DatUtil v1.20

- New resource ROMs in MAME listinfo are automatically detected.
- ClrMamePro log and Nebula driver loaders now use the STORE_ROM
macro (this makes -g work correctly, amongst other things).
- Improved -m (fix merging) behaviour for resource ROMs.
- Nebula drivers now output the correct 'romof' for Neo-Geo clones.

22 April 2002 - DatUtil v1.19

- Added support for the PGM resource.

20 April 2002 - DatUtil v1.18

- Increased maximum number of samples to 10,000 (for PinMAME).
- Exceeding the sample limit now results in a graceful exit.

4 April 2002 - DatUtil v1.17

- Made 'fix-merging' option check the CRC as well as the name.
- Fix merging option warns of CRC conflicts it finds. Useful test!
- Strips trailing spaces from Nebula ROM names (for convenience).
- Executable is called 'datutil.exe' again.

29 March 2002 - DatUtil v1.16

- Added Nebula driver support (useful for data file validation).
- Supports the new '0x' syntax of CMPro logs.
- ZIP scanner now removes '.Zip' from filenames (case sensitive).
- Increased maximum line length to 16384 bytes (for MAME history).
- Increased maximum number of ROMs to 75000.
- Exceeding the game or ROM limits now results in a graceful exit.
- Compiled with zlib v1.1.4

10 February 2002 - DatUtil v1.15

- Enhanced Calice support to cope with Neo-Geo sound ROMs.
- Modified source to compile easily with CygWin (and FreeBSD?).

11 November 2001 - DatUtil v1.14

- Added knowledge of ng-lo.rom.

27 September 2001 - DatUtil v1.13

- Supports CPSE driver file.
- Added 'ignore funny size' option (-z) for non-arcade dats.
- Added 'make lower case' option (-l). Affects names and CRCs.
- Puts "s around empty CMPro header items.
- Fixed directory/zip scanner for ROM names containing spaces.
- Fixed zip scanner to strip uppercase '.ZIP' from game names.
- Improved the 'funny size' warning relating to RomCenter 2 dats.

28 August 2001 - DatUtil v1.12

- Supports new ROM loading introduced in Calice v0.3.8 to v0.4.4.

28 June 2001 - DatUtil v1.11

- Added some warnings for bad quality RomCenter dats
(wrong version number in header and missing ROM sizes).

23 June 2001 - DatUtil v1.10

- Fixed detection of RAINE32 dat format.
- Improved ZIP scanner (not case sensitive when searching for zips).

8 June 2001 - DatUtil v1.9

- Added directory scanning feature (reads ZIPs and subdirectories).
- Calice loader now uses the main driver code, not 'verifyroms'.
- RomCenter 2 loader fixes missing 'romof' details automatically.

18 May 2001 - DatUtil v1.8

- Fixed a minor bug in the '-m' feature (bug introduced in v1.7).
- Now compiled with MinGW (no long filename problems on NT now).

26 April 2001 - DatUtil v1.7

- Added handling of decocass resource in MAME dats.
- Added load/save support for CMPro element 'clrmamepro' and ROMCenter headers.
- Added preliminary support for Calice Driver files.
- Added load support for ClrMamePro log files (i.e. to convert into a dat format).
- Further improved the '-m' feature ('fix merging' option).
- RomCenter2 save routine strips redundant 'merge' information from non-clones.
- DJGPP compilation now uses -Wall option.
- Can now be compiled with Microsoft Visual C++.

22 February 2001 - DatUtil v1.6

- Added option to improve MAME's merging information ('-m').

6 February 2001 - DatUtil v1.5

- Some handling of the playch10 and cvs resources in MAME dats.
- RomCenter 2 is now the default save format when converting CMPro dats.

18 December 2000 - DatUtil v1.4

- Mimic HWC support.
- New format to list game titles ('-f titlelist').
- Year/Manufacturer support for formats that can include them.
- Can specify values for ROMCenter/ROMCenter2 headers.
- Remove clones factility ('-r').
- GWC loader now understands '.speech' tags (System16 Beta).
- Callus95 Patch renamed 'directory' to 'parent'. Now supported.
- Auto sorts when converting to RomCenter format or a Title List.
- Improved sorting (ensures each parent precedes it's clones).
- RAINE and MESS files are converted to CMPro format by default.
- Fixed a problem with the '-g' option and RAINE gameinfo files.
- Increased maximum ROM size to 9 characters (~1GB) for U64Emu.
- Increased number of lines scanned when identifying CMPro dats.
- Source: Added a makefile and included a SAVE_ONLY_FORMAT.
- Source: Changed parameter passing (uses a structure now).
- Preliminary: 'Verify Merging' facility (currently disabled).

19 May 2000 - DatUtil v1.3

- RomCenter 2 support.
- New method of sorting (useful for RomCenter).
- Two EXEs. This is because the long ROM name version eats memory
and is slow on machines without much physical memory.
- Improved handling of invalid parents.
- Warns of funny ROM sizes.
- Generates log of warnings during conversion.

28 April 2000 - DatUtil v1.2

- Increased game/rom name length to 80 characters.
That's about it!

8 April 2000 - DatUtil v1.1

- Added support for JFF.TXT, Sparcade RAT and JAS Romlist.
- Increased maximum number of games from 4096 to 5000.
- Increased game/rom name length to 50 characters.
- Improved listinfo parser so that it can handle odd formatting.
- Other minor modifications.

1 April 2000 - DatUtil v1.0

- Initial version.