Post by dlevere on Aug 4, 2011 23:56:53 GMT -4
This program is used to help find character strings in data files that don't use ASCII encoding. It is primarily for console ROMs, but it works just as well to find any character data in any file type. It's more powerful than a typical relative search because it can find data strings even if they are in a completely arbitrary encoding system.
A typical relative search uses the alphabet as a guide. It assumes that the encoded alphabet is in an incrementing order. So if a=1, b=2, c=3, ... it will find a character string just fine. However, if the alphabet is in a different order, like backwards (a=26, b=25, c=24, ...) the relative search won't find the string. Instead of using the alphabet as a guide, this program uses the searching string.
For example, take the NES game Life Force. The title screen shows the text: "KONAMI INDUSTRY CO". This string has many duplicate characters (O, N, I, etc.). The program creates a binary mask from the search text and uses that to search from. It assumes that all matching characters must match in the binary sense, and all unmatching characters must not match. By doing this you can find the data you're looking for, even if the encoding is arbitrary (as is in the case of Life Force). If you search for this string in a relative search you won't find it, but using this program you will. When you create a table from this you'll find that F=6, I=7, but there is no letter G. This is why the relative search came up empty, because it assumed that the letter G would be there.
To use the program, run it, choose a file to search from. Then type in what you want to search for. Type the search string exactly as it appears in the game. Then click search. The program will give you every match it found. Try to use long search strings in order to limit the total number of matches. Once you've got your match, choose a table file to output to and click the generate table button. This will make you a table using the characters in your search string. Because the program assumes that the encoding is arbitrary, it will -not- create an entire alphabet for you. You should be able to extrapolate the alphabet from the outputted table.
Note: This program only works on the premise that the text is encoded in 8-bit characters with a 1-to-1 text conversion. Some ROMs use 7-bit or 16-bit text encoding or others use a special lookup table (Final Fantasy, Startropics, etc.) which can't be found using this program.
Arbitrary Search
A typical relative search uses the alphabet as a guide. It assumes that the encoded alphabet is in an incrementing order. So if a=1, b=2, c=3, ... it will find a character string just fine. However, if the alphabet is in a different order, like backwards (a=26, b=25, c=24, ...) the relative search won't find the string. Instead of using the alphabet as a guide, this program uses the searching string.
For example, take the NES game Life Force. The title screen shows the text: "KONAMI INDUSTRY CO". This string has many duplicate characters (O, N, I, etc.). The program creates a binary mask from the search text and uses that to search from. It assumes that all matching characters must match in the binary sense, and all unmatching characters must not match. By doing this you can find the data you're looking for, even if the encoding is arbitrary (as is in the case of Life Force). If you search for this string in a relative search you won't find it, but using this program you will. When you create a table from this you'll find that F=6, I=7, but there is no letter G. This is why the relative search came up empty, because it assumed that the letter G would be there.
To use the program, run it, choose a file to search from. Then type in what you want to search for. Type the search string exactly as it appears in the game. Then click search. The program will give you every match it found. Try to use long search strings in order to limit the total number of matches. Once you've got your match, choose a table file to output to and click the generate table button. This will make you a table using the characters in your search string. Because the program assumes that the encoding is arbitrary, it will -not- create an entire alphabet for you. You should be able to extrapolate the alphabet from the outputted table.
Note: This program only works on the premise that the text is encoded in 8-bit characters with a 1-to-1 text conversion. Some ROMs use 7-bit or 16-bit text encoding or others use a special lookup table (Final Fantasy, Startropics, etc.) which can't be found using this program.
Arbitrary Search