Skip to content

Instantly share code, notes, and snippets.

@osyu

osyu/gt_docs.md

Last active October 16, 2024 19:24
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save osyu/895c38cf77f08853353988780e52dfc0 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save osyu/895c38cf77f08853353988780e52dfc0 to your computer and use it in GitHub Desktop.
Download ZIP
Ghost Trick remaster docs
Raw
gt_docs.md

(guides assume you're on Windows)

How to extract game assets

short answer

long answer

primer:

  • Ghost Trick runs on Capcom's RE Engine. REtool is an unofficial tool for dealing with the engine's file formats
  • all resources are stored in a big file named re_chunk_000.pak located in the game's installation directory (which is accessible in steam through [right click] > Manage > Browse local files)
  • unlike a typical archive format like .zip/.rar/.7z, .pak files don't have filenames plainly stored in a way that REtool can reliably retrieve, so you also need to give it a list of names to read from so it knows what to name the things it's extracting. if you don't, you'll be stuck with a bunch of files without discernable names and your life will become harder

what to do:

  • download RETool from the link above and extract REtool.exe to the game's installation directory (the same folder as re_chunk_000.pak)
  • download the filepath list from the link above, into the same folder
  • open a command prompt in that folder. there are a billion ways to do this but the easiest way is typing cmd into the address bar in file explorer
  • type retool -h GTPD_PC_Release.list -x re_chunk_000.pak to extract everything. when it's done there'll be a folder named re_chunk_000 with all the stuff
  • you'll notice there are still hundreds of files without proper names (that look something like 15349567-2389623703.bin). this is normal and most of those files are unimportant/not unique to Ghost Trick, so you can ignore them. it happens because the list we passed to REtool has most but not all filenames, so anything missing will end up with a name like that

How to convert game assets to common formats

(everything here assumes you've just done all the things above)

.tex (textures)

  • type for /r %f in (*.tex.*) do retool -tex %f. this will recursively convert every .tex file to .dds, which is readable by image editing programs like paint.net

.asrc (audio)

  • they're just .wav files with an extra header in front. you have two options:
    • if you have Python installed, run this script to convert to and from the format
    • open up a hex editor (e.g. HxD) and remove the first 78 (0x4e) bytes, then change the file extension to .wav

.mov (video)

  • they're just .wmv files. change the file extension to .wmv and you're set
  • except for natives/stm/streaming/movie/igt_op_480x320_500kbps.mov.1, that's an .mp4

.msg (text)

  • download REMSG_Converter, and type for /r %f in (*.msg.*) do remsg_converter %f to convert recursively to .csv, which you can open in excel or any text editor. add -m json to use json instead

.oft (fonts)

  • these are encrypted OpenType and TrueType font files. you can encrypt/decrypt them using REE.FontsCryptor from the REE.PAK.Tool repository (as of writing binaries aren't provided, you'll have to build it yourself)

.user/.scn/.pfb (objects, scenes, prefabs)

  • you can use the RE_RSZ 010 Editor template to read and edit these

Where the hey is everything

(this is a non-exhaustive list)

textures

  • facepics/portraits: natives/stm/charfiles/3d/08_face
  • backgrounds: natives/stm/charfiles/2d/02_bg
  • character animations: natives/stm/charfiles/2d/01_human
  • object animations: natives/stm/charfiles/2d/03_gimmick
  • ui: natives/stm/charfiles/3d/06_id

audio

  • bgm:
    • remastered: natives/stm/streaming/sound/bgm_new
    • original: natives/stm/streaming/sound/bgm_resource
  • sound effects: natives/stm/streaming/sound/se_resource

video

  • cutscenes: natives/stm/streaming/gui/ui010500/tex
  • unused trailer: natives/stm/streaming/movie

text

  • it's all in natives/stm/gamedesign/text

fonts

  • they're in natives/stm/gui/font/outlinefont

Internal names

stages

  • Junkyard - st01
  • Pigeon Man's Office - st02
  • Prison - st03
  • The Chicken Kitchen - st04
  • Lady's Red Apartment - st05
  • Lynne's Apartment - st06
  • Kamila's Old House - st07
  • Office of the Troubled Man - st09
  • Drive to Chicken Kitchen - st10
  • Special Investigation Unit - st11
  • Temsik Park - st13
  • The Yonoa - st14
  • Prologue & Credits - st15

characters

  • Sissel (human) - cicel
  • Sissel (cat) - neko (猫, lit. "cat")
  • Lynne - linne
  • Ray - kuneri
  • Nearsighted Jeego - killera
  • Commander Sith - sisu
  • Masked Muscleman - migi (右, lit. "right")
  • "One Step Ahead" Tengo - killerb
  • Kamila - girl
  • Missile - pome ("pome[narian]")
  • Emma - fujin (婦人, lit. "madam")
  • Amelie - eimin (永眠, lit. "death")
  • Justice Minister - daijin (大臣, lit. "minister")
  • Detective Mccaw - keijig (刑事, lit. "detective g[reen]")
  • The Blue Detective - keijib (刑事, lit. "detective b[lue]")
  • Odd Blue Doctor - doc
  • Inspector Cabanela - kaba
  • Detective Rindge - caron
  • Guardian of the Park - birakubari (ビラ配り, lit. "handing out leaflets")
  • Pigeon Man - kanri (管理, lit. "management")
  • Chicken Kitchen Cook - cock
  • Typical Cop - keikan (警官, lit. "policeman")
  • Officer Bailey - keibia (警備, lit. "guard a")
  • Rock Jailbird - rock
  • Curry-loving Jailbird - shujinb
  • Detective Jowd - jodo
  • SIU Chief - bucho (部長, lit. "department head")
  • Memry - waitress
  • Dandy - yukaib
  • Beauty - yukaia
  • Chicken Kitchen Bartender - barten ("barten[der]")
  • Alma - mama
  • Minor Crew Hand - zako (雑魚, lit. "small fry")

Modding tools

  • Fluffy Mod Manager - mod manager for RE Engine games (for modifying assets)
  • REFramework - scripting and mod framework (for modifying game logic)
@niltwill
Copy link

niltwill commented Jan 29, 2024 •
edited
Loading

Thanks for the clarification, struct, and the updated script. Only did some shoddy guesses that didn't pay off with unk0 and others.

What's best practice now? Due to these unknown values that cannot be calculated, does one still have to copy these bytes from the original file when re-encoding a modified file? That's a bit cumbersome with batch operations. But that would be the way to go, right? So that nothing breaks, since there is no telling what they do.

It would be great if the script could also take an additional (optional) input (parameter) for the original .asrc audio file (path) with the encoding function (to fetch all unknown data it cannot generate: id, unk0, urate, markers [if any], unk1). That would make it easier for batch operations (rather than manually hex editing every file one by one). Is it possible to pull that off somehow? I will look into it myself eventually, though I'm bad with Python.

Also, did you test the new script? There's a new error that broke it when trying to handle any file now (not only with the problematic files).

Now I get the following error (line 199, in info):

    mark = ','.join(':'.join(str(y) for y in x) for x in mark)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not iterable

Edit: Removed my temporary unclean fix for this after the script update (to make sure it's not getting used in case someone else's reading this).

I tested it with all the aforementioned problematic files, and now it's all OK. Hopefully no other errors now! Yay.

@osyu
Copy link
Author

osyu commented Jan 29, 2024 •
edited
Loading

Oops, again... that line formats the marker list before it gets printed out in the info command, but I forgot to skip that step if it's None (i.e. there are no markers); should be fixed now. I should have caught that but that's what I get for working on things at midnight

You should copy all the unknowns using the info command for now; other things like markers also need to stay, if they exist (you have to have the same amount in your new .asrc or the game script will be liable to break).

I already had plans for an r replace command to do what you're asking for batch operations, I'll work on that next

@osyu
Copy link
Author

osyu commented Jan 29, 2024 •
edited
Loading

Replace command is done; it takes in the input .wav, a base .asrc and an optional output path (if not specified it overwrites the base), plus the loop points and markers. It'll complain if the base has markers and you didn't specify any (or vice versa), so make sure to adjust those for the new audio.

@osyu
Copy link
Author

osyu commented Jan 29, 2024 •
edited
Loading

Updated once more to fix a slew of other bugs, make the out argument required for all commands (to avoid overwriting files you didn't mean to + the splitext stuff wasn't working properly with language suffixes), and add a -cpb option to the replace command which copies the loop positions and markers from the base file rather than using the -lps/-lpe/-mark arguments (useful for batch converting)

@niltwill
Copy link

niltwill commented Feb 22, 2024

AJ will convert to the DS bytecode format, which can be edited with e.g. phoenixtools.

You get no actual text, only integers using that (with scripting\script-dis from that repo). Here is an example for "sc0_0_op.user.2.en.bin" (after decoding it with your script):

SECTION 0
 0000	text "{57473}"
 0001	music 57442, 00001
 0004	newline
 0005	text "{57473}"
 0006	sound 57451, 00001
 0009	section_setup
 0010	hideperson
 0011	text "{57433}"
 0012	cmd62
.label0_15:
 0013	text "{00610}{57473}"
 0015	fullscreen_text
 0016	text "{57435}"
 0017	section_setup
 0018	section_setup
 0019	section_setup
 0020	text "{57433}"
 0021	cmd62
 0022	text "{00609}{57340}\n{57473}"
 0026	finger_choice_2_args_jmp 57323, 4294967172
 0029	newline
 0030	hideperson
 0031	text "{57433}"
 0032	cmd62
[...]

For reference, it's supposed to look something like the attached image in this post.

My guess is that the AJ:AA Trilogy uses UTF-16LE encoding for the text itself, which is why it doesn't work? Other than manual editing with a hex editor (you can sort of read the text parts in these files), there's no easy way right now...

@osyu
Copy link
Author

osyu commented Feb 22, 2024

Yeah, it appears that AJ uses a different encoding here than the DS games, which isn't entirely unexpected since the Unity version of the original trilogy also changed the format some. It segfaults on certain scripts as well (e.g. sc0_0_h00) so it might need more work than just fixing the text parsing. I don't have time right now to look into it further through

@mentionmenot
Copy link

mentionmenot commented Apr 10, 2024 •
edited
Loading

Thank you for the helpful information. Could you provide any update info about decoding AJ GS4 script? I'm encountering challenges with it in my localization project.

@niltwill
Copy link

niltwill commented Apr 10, 2024 •
edited
Loading

Update: Now the script should work with every GS4 binary (I also tested every language).

I made a Python script for the GS4 binary script files. It's not too pretty and the text is hard to edit with it, but it's a start. Also remember that the GS4 script binaries still need to be decoded with this converter first.

Some examples:

ajaat-gs4-script.py decode action_studio.user.2.en.bin
ajaat-gs4-script.py encode action_studio.user.2.en.txt
ajaat-gs4-script.py compare action_studio.user.2.en.bin action_studio.user.2.en.encoded.bin

It supports wildcard matching for encoding and decoding, so you can simply use these commands (while having every script binary in the same folder):

ajaat-gs4-script.py decode *.bin
ajaat-gs4-script.py encode *.txt

With that, the decoding will simply use the same filename with the ".txt" extension.
The encoding will add the ".encoded" prefix (so that you end up with a file like "[name].encoded.bin". In this way, you don't accidentally overwrite the existing binaries, especially since a byte comparison is recommended to make sure there are no discrepancies.

However, there are different problematic files with different languages that need some byte correction after encoding. For English, these are the following:

  • sc0_1_h02.user.2.en.bin
  • sc1_0_000.user.2.en.bin
  • sc1_3_h03.user.2.en.bin
  • sc2_0_013.user.2.en.bin
  • sc2_0_016.user.2.en.bin
  • sc2_3_h00.user.2.en.bin
  • sc3_0_00a.user.2.en.bin
  • sc3_3_00c.user.2.en.bin
  • sc3_3_010.user.2.en.bin

To fix those byte differences that only appear in these 9 files, you can use a command like this: ajaat-gs4-script.py compare --fix sc0_1_h02.user.2.en.bin sc0_1_h02.user.2.en.encoded.bin

(Important to put the original binary file first, then the edited, re-encoded binary.)

Or you can use this batch file, as different languages do have other files that need the byte correction, so it's better to run this through each of these files.

@evil-trainer
Copy link

evil-trainer commented Jul 22, 2024 •
edited
Loading

This gist is for the PC version only, you can find info on modding console versions of RE Engine games elsewhere

Sorry to hijack the gist, but I'm a fan translator of Switch games and unfortunately there isn't somewhere else to look, for addressing the Switch tex conversion situation (at least I couldn't find anything anywhere).

But, hope is not lost, since there's a workaround =)

Thanks to the awesome folks from the Modding Haven Discord Server, I learned how to make MHR_Tex CHopper work with Switch textures.

Open up the texture in a hexadecimal editor and replace the 5º byte with the hexadecimal equivalent of "23", which happens to be 1C.

Before:
image

After:
image

Now, save your changes and drag and drop the texture onto MHR_Tex Chopper and it'll do its thing ;)

image

image

Don't forget to disable "Create MipMap" in Paint.Net BEFORE converting back from .dds to .tex. If you are dealing with UI elements textures, use BC7 linear as the compression or use BC7 sRGB if you're dealing with albedo textures instead. To convert from dds back to .tex, just drag and drop yout edited .dds onto MHR_Tex Chopper and it'll spit out a .tex file. Now, just replace the vanilla .tex with your newly edited one and that's it!

All that's left is create a .pak.patch file and test your edit in-game!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment