lfnetwork.com mark read register faq members calendar

Thread: Jade Empire modding (see first post for info summary)
Thread Tools Display Modes
Post a new thread. Add a reply to this thread. Indicate all threads in this forum as read. Subscribe to this forum. RSS feed: this forum RSS feed: all forums
Old 04-17-2007, 08:17 PM   #1
stoffe
Mostly dormant
 
stoffe's Avatar
 
Status: Administrator
Join Date: Apr 2002
Posts: 5,834
Helpful! 10 year veteran! Notable contributor 
Jade Empire modding (see first post for info summary)

Index - useful resource links:

JE Modding tools (so far):
  • TalkEd v1.1.1a - TLK V4.0 editor for Jade Empire (for modifying all game text in dialog.tlk)

  • ERF/RIM Edit v0.3.6a - ERF/MOD/RIM Editor for Jade Empire (with correct file type associations)

  • JEFindRes v0.3a - Search tool to look for files with names matching the search pattern within all the game's BIF, MOD and RIM files (and the override folder for completeness). Allows extraction of found files and converting images from TXB format to TGA. (See this post for more info)

  • Txb2Tga v0.1a - Converts TXB textures to a TGA and TXI file to allow viewing and modifying textures and images.

  • JE DLGEditor v2.3.0a2 - Dialog editor for Jade Empire, alpha release (See this post for more info )

  • Jade Empire Savegame Editor v0.6 - A basic savegame editor for Jade Empire (see this post for more info)

  • SAC Utils - A save.sac ripper and mender in case you want to try manual GFF editing of your savegame.

Other Modding tools that work with JE:
  • Scripts can be compiled with the KOTOR-adapted versions of nwnnsscomp using nwscript.nss from JE's override folder.

  • 2DAs can be edited with any text editor if they're in 2DA V2.0 format. If they are in the 2DA v2.b binary format they can be edited with KotorTool or tk102's Excel Add-in or converted to 2DA V2.0 with my Convert2da utility.

  • K-GFF v1.2.0 and above seems to work to edit the GFF files from Jade Empire. Make sure you don't use an older version since the new data type JE uses would not be supported then, resulting in data corruption when saving.


Modding info, tutorial and how-to posts:

Released JE Mods in this thread:


* * *



Original post:

Since I just got Jade Empire for PC I thought I'd try to update my RIM/ERF editor to handle the Jade Empire files to make modding it a little easier. The ERF and RIM file formats seems to be mostly the same, though the ResType <--> File extension lookup table seems to have changed, both changing the meaning of some existing ResTypes, and adding a bunch of new file types to it.

I've been looking through various RIM and MOD files so far trying to figure out what the different files are. So far I've only been able to figure out this fairly short list. Do anyone know what other file type <--> ResType associations there are, or what the new files are used for?

Edit: Found a list, so hopefully the table should be complete now. Still need to figure out the purpose of some of the files, and what format they are in though. (Some may be leftovers from Neverwinter Nights / Knights of the Old Republic and unused in Jade Empire.)

Code:
ResType Type    Comment/speculation
------- ----    ---------------------------------------
0       RES     ???
1       BMP     Bitmap image
2       MVE     ???
3       TGA     Targa image file
4       WAV     Wave sound
5       WFX     Wave extra info
7       INI     Configuration file (plaintext)
8       MP3     MP3 sound
9       MPG     MPG video
10      TXT     Text file
11      PVD     Binary data (Pointer to Void Data).

2000    PLH     Polyhedron
2001    TEX     Texture
2002    MDL     Model data (MDL) [used along with MDX file with the same name]
2003    THG     Thing (???)
2005    FNT     Font
2007    LUA     Lua script
2008    SLT     Slot file
2009    NSS     Script sourcecode (plaintext)
2010    NCS     Script bytecode (NCS)
2011    MOD     Module (ERF)
2012    ARE     Static area info (GFF)
2013    SET     Tileset file
2014    IFO     Module information (GFF)
2016    WOK     Area Walkmesh
2017    2DA     Data table (plaintext or 2DA V2.b binary)
2018    TLK     Talk table file (TLK)
2022    TXI     Extra Texture Info (plaintext)
2023    GIT     Game Instance File (dynamic area info) (GFF)
2024    ITM     Item template (GFF)
2026    CRE     Creature template (GFF)
2029    DLG     Dialog file (GFF)
2030    PAL     Palette (GFF)
2031    TRG     Triggers (GFF)
2033    DDS     DXTC Compressed texture
2034    SND     Sounds (???)
2036    LTR     ???
2037    GFF     Generic GFF file (GFF... duh)
2038    FAC     Faction file (GFF)
2039    ENC     Encounter template (GFF)
2040    CON     Container template (GFF)
2041    DOR     Door template (GFF)
2043    PLA     Placeable template (GFF)
2045    DFT     ???
2046    GIC     Game Instance Comments (area editor comment data)
2047    GUI     Graphical User Interface definition file. (GFF)
2048    CSS     Conditional Source Script
2049    CCS     Conditional Compiled Script
2050    MER     Merchant/store template (GFF)
2052    DWK     Door walkmesh
2053    PWK     Placeable walkmesh (plaintext)
2054    GEN     Random Item Generator
2056    JRL     Journal file (GFF)
2057    SAV     Area game state data (GFF)
2058    WAY     waypoint template (GFF)
2059    4PC     Custom 16-bit RGBA texture format
2060    SSF     sound set file (SSF)
2061    HAK     HAK file (ERF)
2062    NWM     NWN Official Campaign module (ERF)
2063    BIK     BINK movie (Bink video)
2064    MIN     MiniGame data (GFF)
2065    NDB     Script debugger data (toolset)
2066    XWB     XACT WaveBank
2067    XSB     XACT SoundBank
2068    BIN     XBOX DSP filter

3000    LYT     Area model layout (plaintext)
3001    VIS     Area visibility graph (plaintext)
3002    RIM     Resource Image File (RIM)
3004    LIP     Lipsync Visemes data
3005    XMV     ???
3006    WMA     ???
3007    CWD     Crowd game object template (GFF)
3008    PRO     Projectile static resource type
3009    AOE     Area of Effect static resouce type
3010    MAT     material file (plaintext)
3011    MAB     material file (binary)
3012    QST     Quest/journal data (GFF)
3013    STO     Store static resource file (GFF) [used with MER file?]
3014    APL     Artist placeable
3015    HEX     Hex grid (plaintext) [appears to be map-related in some way]
3016    MDX     Extra model data (binary) [used along with MDL file]
3017    TXB     Compressed image with TXI data
3018    TPC     Compressed image with TXI data
3019    CAM     ???
3020    SPT     ???
3021    STYLE   ???
3022    FSM     Finite State Machine (FSM) data
3023    ART     [Weather/Light data for area?] (INI/plaintext)
3024    AMP     ??? (binary)
3025    CWA     Crowd Attribute (GFF)
3026    XLS      MS Excel spreadsheet
3027    SPF     NGF: Style profiles
3028    BIP     PATL: Binary LIP. - [lipsync/Sound file?]

9996    IDS     Legacy BG Script File
9997    ERF     Encapsulated file
9998    BIF     BIFF file
9999    KEY     Key file

Last edited by tk102; 01-08-2008 at 03:52 AM. Reason: Update JSE to v0.6
stoffe is offline   you may: quote & reply,
Old 04-18-2007, 04:09 AM   #2
oldflash
Veteran
 
oldflash's Avatar
 
Join Date: Aug 2004
Posts: 904
Current Game: SWtOR
Forum Veteran Veteran Modder Helpful! 
Quote:
Originally Posted by stoffe
Since I just got Jade Empire for PC I thought I'd try to update my RIM/ERF editor to handle the Jade Empire files to make modding it a little easier. The ERF and RIM file formats seems to be the same, though the ResID <--> File extension lookup table seems to have changed, both changing the meaning of some existing ResIDs, and adding a bunch of new file types to it.
That's a good news.
I've done few things like editing some 2da's and make some scripts.
For that I have use 2DA Converter/Merger Tool, ERF/RIM Editor and KotorTool.
Maybe you have a look into .tlk files? All I want is to set description for few items.
oldflash is offline   you may: quote & reply,
Old 04-18-2007, 11:40 AM   #3
stoffe
Mostly dormant
 
stoffe's Avatar
 
Status: Administrator
Join Date: Apr 2002
Posts: 5,834
Helpful! 10 year veteran! Notable contributor 
Quote:
Originally Posted by oldflash
Maybe you have a look into .tlk files? All I want is to set description for few items.
I've made a new version of TalkEd that should handle the TLK V4.0 format used by Jade Empire. It seems to work to read and modify the English TLK file with it from my quick testing. Though please keep an eye out for bugs if you use it since I've only had time to do very limited testing. You can download it here for now. (This version only works with the Jade Empire TLK files, not the KOTOR ones.)

There is one data field in the TLK file I don't know the purpose of currently. It seems to be a 32 bit unsigned integer which is set to 0xFFFFFFFF (-1 I suppose) for the majority of the TLK entries, though a fair number of them (mostly dialog related strings) has some other value set here, and it seems each value (except 0xFFFFFFFF) used is unique and set for just that one entry.

TalkEd just lets you set this number directly (using hexadecimal notation) for the time being until someone can figure out what the purpose of this data is.

Edit: I suspect the unknown numeric value refers to names of the numbered BIP/sound files found in for example audio.bif, though I haven't been able to verify this yet.


mt

Last edited by stoffe; 05-06-2007 at 11:41 AM.
stoffe is offline   you may: quote & reply,
Old 04-18-2007, 12:04 PM   #4
tk102
Well past expiration date
 
tk102's Avatar
 
Join Date: Jan 2004
Posts: 5,767
Current Game: FTL
Forum Veteran Helpful! Notable contributor 
stoffe, have you tried poking around in the .key/.bif files to fill in the gaps for erf extensions? I found

this thread
on Bioware's forums that references a tool that recognizes most of the key/bif extensions for Jade Empire. Just a thought.


tk102 is offline   you may: quote & reply,
Old 04-18-2007, 12:09 PM   #5
stoffe
Mostly dormant
 
stoffe's Avatar
 
Status: Administrator
Join Date: Apr 2002
Posts: 5,834
Helpful! 10 year veteran! Notable contributor 
Quote:
Originally Posted by tk102
stoffe, have you tried poking around in the .key/.bif files to fill in the gaps for erf extensions? I found
[url=http://jade.bioware.com/forums/viewtopic.html?topic=554641&forum=108&highlight=er f
this thread[/url] on Bioware's forums that references a tool that recognizes most of the key/bif extensions for Jade Empire. Just a thought.
I haven't, but the problem I can see with that is that the KEY/BIF format doesn't store file extensions either, as far as I am aware. So if the lookup table for ResID <--> File type has changed for Jade Empire tools made for other games would still output data with the wrong file extension, or no valid extension at all, since it has no way of knowing what those ResID's are supposed to be.

That the data files seems to be an even more disorganized mess than KOTOR, with files or the same type or files belonging to the same area scattered all over the place, doesn't make it easier to find all the file types that are used and how they fit together
stoffe is offline   you may: quote & reply,
Old 04-18-2007, 12:16 PM   #6
tk102
Well past expiration date
 
tk102's Avatar
 
Join Date: Jan 2004
Posts: 5,767
Current Game: FTL
Forum Veteran Helpful! Notable contributor 
It was my hope that the tool referenced above (in a broken url link, now fixed) would shed the light on some unknown extensions. A long shot I'm sure, but just a different angle to attack from.
tk102 is offline   you may: quote & reply,
Old 04-18-2007, 03:30 PM   #7
stoffe
Mostly dormant
 
stoffe's Avatar
 
Status: Administrator
Join Date: Apr 2002
Posts: 5,834
Helpful! 10 year veteran! Notable contributor 
Quote:
Originally Posted by tk102
It was my hope that the tool referenced above (in a broken url link, now fixed) would shed the light on some unknown extensions. A long shot I'm sure, but just a different angle to attack from.
Poked around the BIF files a bit, and while they do suffer from the same problem of incorrect/unknown file extensions I managed to figure out a few more things (updated the list in the first post).

I've made a JE version of my ERF/RIM Editor that uses the above file extension/ResType table (minus the couple of unknowns in the list) so far. While the list isn't complete it's easier to spot which files aren't properly identified when it doesn't use the KOTOR file associations.

If anyone is interested you can download it from the link in the first post in this thread. If you notice any file types that are listed starting with a # followed by a number those are of unknown types. Please let me know what file it it and where (in what file) you found them in those cases.


mt

Last edited by stoffe; 04-28-2007 at 06:25 AM.
stoffe is offline   you may: quote & reply,
Old 04-19-2007, 05:19 AM   #8
oldflash
Veteran
 
oldflash's Avatar
 
Join Date: Aug 2004
Posts: 904
Current Game: SWtOR
Forum Veteran Veteran Modder Helpful! 
That fiels is for sound ("0xFFFFFFFF" value = no sound). I test this by replacing values and it works. However the sound file must be loaded in that module or to be present in override because if I set a sound which is used in other module will do nothing. I haven't test append new entry but I'm sure is no problem with that. This tool is perfect for what I need to finish my mod.
Many thanks stoffe.
Edit: I just found dialog tool and I should test-it tonight.
I like how easy is to attach scripts for diferite events by editing ar_*.2da files.

Last edited by oldflash; 04-19-2007 at 06:11 AM.
oldflash is offline   you may: quote & reply,
Old 04-19-2007, 12:08 PM   #9
stoffe
Mostly dormant
 
stoffe's Avatar
 
Status: Administrator
Join Date: Apr 2002
Posts: 5,834
Helpful! 10 year veteran! Notable contributor 
Quote:
Originally Posted by oldflash
That fiels is for sound ("0xFFFFFFFF" value = no sound). I test this by replacing values and it works. However the sound file must be loaded in that module or to be present in override because if I set a sound which is used in other module will do nothing.
Ok, thanks for the confirmation. I've updated TalkEd to handle the values as sound reference numbers and not display them in hexadecimal notation.

Quote:
Originally Posted by oldflash
I like how easy is to attach scripts for diferite events by editing ar_*.2da files.
Haven't had much time to look at how things fit together yet since I've only had the game installed for roughly a day so far, but that may be very useful if they don't have any debug console in the game (haven't managed to find one)


mt
stoffe is offline   you may: quote & reply,
Old 04-19-2007, 04:41 PM   #10
stoffe
Mostly dormant
 
stoffe's Avatar
 
Status: Administrator
Join Date: Apr 2002
Posts: 5,834
Helpful! 10 year veteran! Notable contributor 
Another update...

While poking around in the RIM and MOD files I found some more file types I managed to figure out with reasonably certainty what they should be. I've updated the list in the first post and uploaded another version of ERF/RIM Edit with the recent changes, see the link in the first post in this thread to download.

Any other file types I've not managed to find yet?

Last edited by stoffe; 04-28-2007 at 06:26 AM.
stoffe is offline   you may: quote & reply,
Old 04-20-2007, 01:57 AM   #11
RedHawke
Shadow Lord Of The Sith™
 
RedHawke's Avatar
 
Status: Moderator
Join Date: Mar 2004
Location: Stormreach CA.
Posts: 9,184
Current Game: DDO, Stormreach
Thanks for all the effort stoffe! =)

Now I just need to get off my lazy behind and install the darn game (DDO is evil... just Evil!)


"Beware the form-fitting black armor-clad Drow hottie with twin Mineral II Greensteel Khopeshes!"
"Liella d'Orien says, '"You're the fool, Devil. -- Witness the power of this fully ARMED and OPERATIONAL Titan!"'"
----------------------------------------------------------------------
RedHawke is offline   you may: quote & reply,
Old 04-20-2007, 03:57 AM   #12
mjpb3
Forumite
 
mjpb3's Avatar
 
Join Date: Oct 2005
Location: Eccentricity
Posts: 632
Kind of on topic here, but if someone could find a way to open the PC & NPC textures (they're in txb form) I would be ever so grateful.

(If this doesn't fit in here, please excuse me. )



.·:*¨¨*:·..·:*¨My Mods¨*:·..·:*¨¨*:·.
Disciple FTW!
There is no canon. There is only Revan.
FYI: I am a girl, and you may call me Janet.
It's all just fun and games until the flying monkeys attack.
mjpb3 is offline   you may: quote & reply,
Old 04-20-2007, 03:13 PM   #13
stoffe
Mostly dormant
 
stoffe's Avatar
 
Status: Administrator
Join Date: Apr 2002
Posts: 5,834
Helpful! 10 year veteran! Notable contributor 
Quote:
Originally Posted by mjpb3
Kind of on topic here, but if someone could find a way to open the PC & NPC textures (they're in txb form) I would be ever so grateful.
The TXB format has some similarities to the TPC format used for the KOTOR games. The header is different, but like TPC it contains compressed bitmaps and optionally TXI data at the end.

Don't know if I'm good enough at this kind of programming to be able to make a converter that converts TXB --> TGA directly, but I'll give it a try and see how it goes.


Quote:
Originally Posted by oldflash
I have no idea how to use a debug console or what is that
I was just looking for an in-game debug console (similar to the yellow "> " you can type commands at in KOTOR1, and the invisible one in TSL), but it doesn't seem like Jade Empire has one. Hopefully most of that can be handled via scripts instead though.
stoffe is offline   you may: quote & reply,
Old 04-21-2007, 06:21 AM   #14
oldflash
Veteran
 
oldflash's Avatar
 
Join Date: Aug 2004
Posts: 904
Current Game: SWtOR
Forum Veteran Veteran Modder Helpful! 
Quote:
Originally Posted by stoffe
I was just looking for an in-game debug console (similar to the yellow "> " you can type commands at in KOTOR1, and the invisible one in TSL), but it doesn't seem like Jade Empire has one. Hopefully most of that can be handled via scripts instead though.
Ah, stupid me... I was thinking to something more complex... I saw a console_font file somewhere.

Great news about textures and I must thank you all for efforts.
oldflash is offline   you may: quote & reply,
Old 04-20-2007, 06:47 AM   #15
oldflash
Veteran
 
oldflash's Avatar
 
Join Date: Aug 2004
Posts: 904
Current Game: SWtOR
Forum Veteran Veteran Modder Helpful! 
@stoffe: many thanks again for all this tools.
I have no idea how to use a debug console or what is that
Quote:
.ndb files are debug information for the script debugger and not necessary for the game to function.
... if this info good for anything.
However, I belive to be possible to send mesajes to log or display by #include debug.

@mjpb3: I also look for this and for a decomp. for mdl files

Last edited by oldflash; 04-20-2007 at 10:40 AM.
oldflash is offline   you may: quote & reply,
Old 04-20-2007, 03:21 PM   #16
tk102
Well past expiration date
 
tk102's Avatar
 
Join Date: Jan 2004
Posts: 5,767
Current Game: FTL
Forum Veteran Helpful! Notable contributor 
I built a TPC Perl mdoule based on eiz's tpc2tga source code. If you want any assistance let me know.


tk102 is offline   you may: quote & reply,
Old 04-20-2007, 04:17 PM   #17
stoffe
Mostly dormant
 
stoffe's Avatar
 
Status: Administrator
Join Date: Apr 2002
Posts: 5,834
Helpful! 10 year veteran! Notable contributor 
Quote:
Originally Posted by tk102
I built a TPC Perl mdoule based on eiz's tpc2tga source code. If you want any assistance let me know.
Thanks, that will probably be needed if this is going to go anywhere at all. I had a look at the tpc2tga source to get an idea of how to decode the DXT data and was immediately reminded why I strongly dislike C++ (couldn't even get that code to compile, so I'm doing it in Delphi instead).

Edit: Nevermind, I was just stupid.

Last edited by stoffe; 04-21-2007 at 12:30 PM.
stoffe is offline   you may: quote & reply,
Old 04-20-2007, 05:08 PM   #18
mjpb3
Forumite
 
mjpb3's Avatar
 
Join Date: Oct 2005
Location: Eccentricity
Posts: 632
Whatever you can come up with would be excellent for me. That's the main thing I ever wanted to mod in Jade Empire--the textures. Thanks for anything you can accomplish.



.·:*¨¨*:·..·:*¨My Mods¨*:·..·:*¨¨*:·.
Disciple FTW!
There is no canon. There is only Revan.
FYI: I am a girl, and you may call me Janet.
It's all just fun and games until the flying monkeys attack.
mjpb3 is offline   you may: quote & reply,
Old 04-20-2007, 05:22 PM   #19
tk102
Well past expiration date
 
tk102's Avatar
 
Join Date: Jan 2004
Posts: 5,767
Current Game: FTL
Forum Veteran Helpful! Notable contributor 
Code:
sub reverse_bytes {
	my $in=shift;
	return unpack('V',pack('C4',reverse (unpack('C4',pack('V',$in)))));
}
It is indeed a reversal of bytes (0xAABBCCDD becomes 0xDDCCBBAA for example).

This is so the $cpx variable in decode_dxt1 can be right shifted 2 bits at a time and disregarded as the new $px is calculated. The blended array is set up in such a way to make sense of the two bits (which are now reversed).

see:
Code:
for (my $y=0; $y<4; $y++) {
  for (my $x=0; $x<4; $x++) {
    my $px = $cpx & 3;
    $pixels[($ty-4+$y)*$pitch +($tx+$x)]=$blended[$px];
    $cpx >>=2;
  }
}

I don't have the tpc2tga source in front of me at the moment so I'm just working off the tpc.pm, btw -- the Perl code was a direct (more or less) translation of the C++.

Edit: crossed out the part about the 2 bits being reversed -- they aren't reversed -- their position within their bytes are unaltered but the bytes themselves are reordered.

Last edited by tk102; 04-20-2007 at 05:44 PM.
tk102 is offline   you may: quote & reply,
Old 04-20-2007, 05:54 PM   #20
stoffe
Mostly dormant
 
stoffe's Avatar
 
Status: Administrator
Join Date: Apr 2002
Posts: 5,834
Helpful! 10 year veteran! Notable contributor 
Quote:
Originally Posted by tk102
Code:
sub reverse_bytes {
	my $in=shift;
	return unpack('V',pack('C4',reverse (unpack('C4',pack('V',$in)))));
}
It is indeed a reversal of bytes (0xAABBCCDD becomes 0xDDCCBBAA for example).
That reminds me why I'm not too fond of Perl either. I haven't done anything with Perl for over 6 years, so I didn't understand much of that code. (I prefer high-level "almost english" languages like Delphi, C#, PHP, so it feels like I'm in way over my head with this.)

The unpack() function takes a string and splits it by token into an array, like explode() in PHP? While pack() takes an array and spits out a token-separated string from it?

What does your reverse_bytes() function receive as parameter, a token-separated string (using "V" as separator) containing the three values a texel-struct holds?

If so you convert the string to an array, immediately explode the array into a string again (this time using "C4" as token separator instead), reverse-sort the string, turn it into an array again (wouldn't "C4" be "4C" when you reverse the string?), and immediately turns the array back into a string (this time separated with "V" again) which you return? Wouldn't that return an array of 3 values?

Huh? I don't follow what this accomplishes. It obviously works, but I still don't understand what it does.

(As you don't have the source: The C++ code takes a struct as function parameter containing two 16 bit unsigned ints and one 32 bit unsigned int value... does something with those three values which I have absolutely no idea what, and then spits out a single 32 bit unsigned integer as result.)

* * *

Another question ... this is called in a loop that loops up to the Y-size of the bitmap while writing the TGA (i is the incrementing value in the loop):
fwrite(file->pixels+pitch*(ysz-i-1), 4, pitch, fp)

file->pixels is an array of 32 bit unsigned integers (the RGBA data), pitch is the X-dimension of the image (in pixels), ysz is the Y-dimension of the image

What exactly is it writing? file->pixels is an array, but it doesn't supply any array index in that function call... Does the file->pixels+pitch*(ysz-i-1) add pitch*(ysz-i-1) to the memory pointer of the pixels array and then write the four bytes found in that location? Why do this instead of using an array index?

Weird lower-level languages I haven't used for years confuse me. Use of pointers always confuse me.
stoffe is offline   you may: quote & reply,
Old 04-20-2007, 06:08 PM   #21
tk102
Well past expiration date
 
tk102's Avatar
 
Join Date: Jan 2004
Posts: 5,767
Current Game: FTL
Forum Veteran Helpful! Notable contributor 
Quote:
Originally Posted by stoffe
The unpack() function takes a string and splits it by token into an array, like explode() in PHP? While pack() takes an array and spits out a token-separated string from it?
No, The pack and unpack functions are Perl's way forcing data to/from certain interpretations like an unsigned long (template: V) or a unsigned byte (template: C). This is needed because of Perl's weak typing mechanisms. pack takes data forces it into a datatype while unpack translates coerced data back into perl scalars.


Quote:
What does your reverse_bytes() function receive as parameter, a token-separated string (using "V" as separator) containing the three values a texel-struct holds?
reverse_bytes accepts a perl scalar (which itself was populated from an unpack function with template V). It then packs this value into an unsigned long again, unpacks it into 4 unsigned bytes, reverses the array of bytes, repacks the array of bytes, and then unpacks the result as unsigned long.

Quote:
(As you don't have the source: The C++ code takes a struct as function parameter containing two 16 bit unsigned ints and one 32 bit unsigned int value... does something with those three values which I have absolutely no idea what, and then spits out a single 32 bit unsigned integer as result.)
Anyway the more elegant C++ code appears as this:
Code:
inline u32 reverse_bytes(u32 i, int2type<4>) {
	return    (i&0xFF000000) >> 24 
		| (i&0x00FF0000) >>  8
		| (i&0x0000FF00) <<  8
		| (i&0x000000FF) << 24;
}
Here you can see better the masking and data shifting involved. The perl code accomplishes the same thing but uses a temporary array of bytes that it reverses and repacks into a long then unpacks to get the numeric value.

* * *

Last edited by tk102; 04-20-2007 at 07:10 PM.
tk102 is offline   you may: quote & reply,
Old 04-20-2007, 07:10 PM   #22
tk102
Well past expiration date
 
tk102's Avatar
 
Join Date: Jan 2004
Posts: 5,767
Current Game: FTL
Forum Veteran Helpful! Notable contributor 
Quote:
Another question ... What exactly is it writing? file->pixels is an array, but it doesn't supply any array index in that function call... Does the file->pixels+pitch*(ysz-i-1) add pitch*(ysz-i-1) to the memory pointer of the pixels array and then write the four bytes found in that location? Why do this instead of using an array index?
Yes that is exactly what it is doing, byte by byte. I believe this syntax was used because in the tpc.cpp file, you can see file->pixels+pitch*(ysz-i-1) is used to assign the values byte-by-byte. I think eiz just used copy/paste of the code as a mnuemonic so he could easily recognize what exactly he looping on.

Last edited by tk102; 04-21-2007 at 12:18 PM.
tk102 is offline   you may: quote & reply,
Old 04-20-2007, 08:20 PM   #23
tk102
Well past expiration date
 
tk102's Avatar
 
Join Date: Jan 2004
Posts: 5,767
Current Game: FTL
Forum Veteran Helpful! Notable contributor 
The fwrite function is writing row by row starting at the end of the array. Think of the array as the image mirrored vertically. (Left-right is maintained but top-bottom is reversed.) I'm sorry for confusing you by saying byte-by-byte earlier.

Here's the declaration for fwrite
Code:
size_t fwrite(const void *ptr, size_t size, size_t nitems,
    FILE *stream);
We are using size=4, and nitems=pitch (the width in pixels of the iamge), so each fwrite is writing 4*pitch bytes. The fwrite function returns the number of items (not bytes) which should be equal to pitch in our case. The array's memory address begins at file->pixels, so at the beginning of the loop to write the last row from the array the offset would be (pitch*(ysz-1)). The next to last row would begin at (pitch*(ysz-2)).

HTH and sorry for not having the concentration earlier.

Last edited by tk102; 04-21-2007 at 12:19 PM.
tk102 is offline   you may: quote & reply,
Old 04-21-2007, 08:16 AM   #24
stoffe
Mostly dormant
 
stoffe's Avatar
 
Status: Administrator
Join Date: Apr 2002
Posts: 5,834
Helpful! 10 year veteran! Notable contributor 
Quote:
Originally Posted by tk102
The fwrite function is writing row by row starting at the end of the array. Think of the array as the image mirrored vertically. (Left-right is maintained but top-bottom is reversed.) I'm sorry for confusing you by saying byte-by-byte earlier.
But still something does not seem right with how I'm understanding this... In the above example each line of the image has 256 pixels... and each pixel is represented by a dword (4 byte integer) holding its RGBA value. In other words each line contains 4 * 256 = 1024 bytes, which you read from the array and write to the file.

But each iteration of the loop still only seems to back off pitch (=256) bytes. Now can it back up one row if it only backs up 25% of the data that is being read/written? Shouldn't it back up pitch*4 to get to the next line of data? Does the memory pointer voodoo deal with array elements and not byte positions in memory? I.e. it backs up pitch (=256) elements and not bytes?

Edit again: Yes! That seems to have done the trick. Seems like the utility works to convert at least those TXB files to TGA I have tested it on.


mt

Last edited by stoffe; 04-21-2007 at 11:49 AM.
stoffe is offline   you may: quote & reply,
Old 04-21-2007, 12:08 PM   #25
tk102
Well past expiration date
 
tk102's Avatar
 
Join Date: Jan 2004
Posts: 5,767
Current Game: FTL
Forum Veteran Helpful! Notable contributor 
\o/ Yay for stoffe! Did you answer your own question then? The *4 part seems like it should be there to me too... but then again I'm not a C++ guru. I would've expected the use of brackets here to denote element iteration. (Don't like having to think at this level with my programming either, stoffe.) I remember this is the area I fumbled through with trial and error also until I got something recognizable. Good job.
tk102 is offline   you may: quote & reply,
Old 04-21-2007, 12:16 PM   #26
stoffe
Mostly dormant
 
stoffe's Avatar
 
Status: Administrator
Join Date: Apr 2002
Posts: 5,834
Helpful! 10 year veteran! Notable contributor 
Quote:
Originally Posted by tk102
\o/ Yay for stoffe! Did you answer your own question then?
More or less, I figured out how to replicate the behavior reasonably using array indexes instead.

The quick converter utility seems to work reasonably well on the few TXB textures I've tried to convert with it. I've uploaded it for now in case someone else wants to help test it and determine how well it works: Txb2Tga.

Use at your own risk. It seems to work OK, but I've only tested it on a handful of different TXB files. If you encounter any file it can't convert properly please let me know which file that was.

* * *

On more general notes it seems the NWNNSSCOMP for KOTOR (at least the variant tk102 did) works fine to compile scripts for Jade Empire if you use the nwscript.nss file found in the override folder and fix the couple of syntax errors it contains.

Jade Empire also seems to use a mix of the traditional plaintext 2DA V2.0 format, and the same binary 2DA V2.b format used with KOTOR. So the binary files can be edited with KotorTool, or converted to text using tk102's excel plugin or my quick converter utility.
stoffe is offline   you may: quote & reply,
Old 04-21-2007, 08:28 PM   #27
mjpb3
Forumite
 
mjpb3's Avatar
 
Join Date: Oct 2005
Location: Eccentricity
Posts: 632
Yes yes yes!!!! Thank you stoffe and tk102! It works! I was able to convert Scholar Ling's outfit and head texture. Just want I needed!




.·:*¨¨*:·..·:*¨My Mods¨*:·..·:*¨¨*:·.
Disciple FTW!
There is no canon. There is only Revan.
FYI: I am a girl, and you may call me Janet.
It's all just fun and games until the flying monkeys attack.
mjpb3 is offline   you may: quote & reply,
Old 04-21-2007, 10:58 PM   #28
AjuntaPall316
Lurker
 
AjuntaPall316's Avatar
 
Join Date: Aug 2006
Posts: 8
Thanks for all your hard work, nowhere else is even close to this far for texture editing yet. But how do you get these new textures into the game now? Do you need to change them back to txb's? If so, how?
AjuntaPall316 is offline   you may: quote & reply,
Old 04-22-2007, 07:50 AM   #29
stoffe
Mostly dormant
 
stoffe's Avatar
 
Status: Administrator
Join Date: Apr 2002
Posts: 5,834
Helpful! 10 year veteran! Notable contributor 
Quote:
Originally Posted by AjuntaPall316
Thanks for all your hard work, nowhere else is even close to this far for texture editing yet. But how do you get these new textures into the game now? Do you need to change them back to txb's? If so, how?
It should work to put both the TGA and TXI file in the override folder directly. Make sure there isn't a corresponding TXB file located in the override folder as well, since it will take precedence over the TGA's. Many of the player appearance textures are located in the override folder as well as in players.mod (where those in override will be used since it has higher priority than the BIF/RIM/MOD files).

I did a quick test and recolored the shirt of my character green, moved p_blossom01.txb out of the override folder and put p_blossom01.tga and p_blossom01.txi in the override folder, and she turned up with a green shirt in-game.

(I.e. you shouldn't need to remove the TXB file from within the data files for the TGA to be used, just make sure one doesn't exist in the override folder.)


Edit: I did a quick reskin of the character you mentioned as well, and it showed up fine in-game:

Last edited by stoffe; 05-08-2007 at 02:06 PM.
stoffe is offline   you may: quote & reply,
Old 04-22-2007, 12:43 AM   #30
tk102
Well past expiration date
 
tk102's Avatar
 
Join Date: Jan 2004
Posts: 5,767
Current Game: FTL
Forum Veteran Helpful! Notable contributor 
I assume the .tga files are readable as-is in the override directory. Bioware has been pretty good about that in the past, just as they do with v2.b and v2.0 2da files.
tk102 is offline   you may: quote & reply,
Old 04-22-2007, 02:10 AM   #31
AjuntaPall316
Lurker
 
AjuntaPall316's Avatar
 
Join Date: Aug 2006
Posts: 8
Still can't get it to work. I converted Radiant Jen Zi's body and edited it, then saved it as TGA image named P_Li01 which is the same as the txb file. I also tried naming it P_Li01.txb which I knew wouldn't work and it didn't. I deleted the actual txb file thinking it might just be loading that one first, and that didn't work either. Her body came up black, so I replaced it. Any Ideas?
AjuntaPall316 is offline   you may: quote & reply,
Old 04-22-2007, 05:33 PM   #32
AjuntaPall316
Lurker
 
AjuntaPall316's Avatar
 
Join Date: Aug 2006
Posts: 8
Thanks, I got it working now. I don't know why it didn't work the last time I got rid of the TXB though. Thanks again.
AjuntaPall316 is offline   you may: quote & reply,
Old 04-23-2007, 10:41 AM   #33
stoffe
Mostly dormant
 
stoffe's Avatar
 
Status: Administrator
Join Date: Apr 2002
Posts: 5,834
Helpful! 10 year veteran! Notable contributor 
GFF editing observations

Seems like Jade Empire has added yet another data type to the GFF format (still without changing the version number ). Type 18 appears to be a StrRef type, similar to ExoLocString in use but only containing a dialog.tlk reference and not any localized substring data. Be warned that these fields may get messed up and stripped if using any GFF editors for NWN or KOTOR to edit Jade Empire's GFF files.

Aside from that it seems JE uses the Rotation and Vector field types added in KOTOR as well, which would cause trouble when using GFF editors made for NWN.
stoffe is offline   you may: quote & reply,
Old 04-23-2007, 11:08 AM   #34
tk102
Well past expiration date
 
tk102's Avatar
 
Join Date: Jan 2004
Posts: 5,767
Current Game: FTL
Forum Veteran Helpful! Notable contributor 
So Field 16 and 17 exist-- that's good. Have you been able to tell if CResRef fields include up to 32 chars (like NWN2) or just 16 chars (like KotOR)? If it helps, I could modify either KGFF or NWN2GFF to include field 18. Send me a PM if you like.
tk102 is offline   you may: quote & reply,
Old 04-23-2007, 01:24 PM   #35
stoffe
Mostly dormant
 
stoffe's Avatar
 
Status: Administrator
Join Date: Apr 2002
Posts: 5,834
Helpful! 10 year veteran! Notable contributor 
Quote:
Originally Posted by tk102
So Field 16 and 17 exist-- that's good. Have you been able to tell if CResRef fields include up to 32 chars (like NWN2) or just 16 chars (like KotOR)? If it helps, I could modify either KGFF or NWN2GFF to include field 18. Send me a PM if you like.
As far as I can tell Jade Empire uses 16 character ResRefs. It seems to be the same format used for KOTOR except for (at least) one new data type that looks like it holds a StrRef from a quick look at it. It would be very useful if you could make a JE-compatible version of k-gff.
stoffe is offline   you may: quote & reply,
Old 04-23-2007, 07:19 PM   #36
tk102
Well past expiration date
 
tk102's Avatar
 
Join Date: Jan 2004
Posts: 5,767
Current Game: FTL
Forum Veteran Helpful! Notable contributor 
Quote:
Originally Posted by stoffe
As far as I can tell Jade Empire uses 16 character ResRefs. It seems to be the same format used for KOTOR except for (at least) one new data type that looks like it holds a StrRef from a quick look at it. It would be very useful if you could make a JE-compatible version of k-gff.
KGFF v1.1.9 now supports StrRef fields (Field Type: 18). I anticipate if/when KotOR3 gets made, there is a good likelihood it will use this field type as well.

@Adelphus: I don't believe cchargin or Taina have devoted time to modifying their modeling tooling for Jade Empire at this point.
tk102 is offline   you may: quote & reply,
Old 04-23-2007, 02:45 PM   #37
Adelphus
Rookie
 
Adelphus's Avatar
 
Join Date: Apr 2007
Location: Vancouver, B.C.
Posts: 53
Is there any current information about decompiling and replacing models at this time?
Adelphus is offline   you may: quote & reply,
Old 04-23-2007, 10:26 PM   #38
Adelphus
Rookie
 
Adelphus's Avatar
 
Join Date: Apr 2007
Location: Vancouver, B.C.
Posts: 53
Nice work on KGFF tk, and thanks for the info.
Adelphus is offline   you may: quote & reply,
Old 04-28-2007, 06:30 AM   #39
stoffe
Mostly dormant
 
stoffe's Avatar
 
Status: Administrator
Join Date: Apr 2002
Posts: 5,834
Helpful! 10 year veteran! Notable contributor 
I've uploaded another minor update to the ERF/RIM Editor, adding some more file associations to the table. See the link in the first post in this thread to download it.
stoffe is offline   you may: quote & reply,
Old 04-30-2007, 07:36 AM   #40
mjpb3
Forumite
 
mjpb3's Avatar
 
Join Date: Oct 2005
Location: Eccentricity
Posts: 632
stoffe, just a couple questions/comments about your TXB2TGA converter...

Not that it doesn't work properly or anything--and I can use it just as it is--but, I was curious as to why I get so much dithering when converting the TXBs to TGAs.

Also, is there any way for the converter to increase the output file size from 256x256 to the more commonly used 512x512? It just seems as if the smaller file size could be causing the dithering, but I could be completely wrong about that and the dithering is caused by something else entirely.

Anyway, thank you so much for this converter! Whether or not you can do anything else with it, it still works fine just as it is and I am enjoying using it a lot!



.·:*¨¨*:·..·:*¨My Mods¨*:·..·:*¨¨*:·.
Disciple FTW!
There is no canon. There is only Revan.
FYI: I am a girl, and you may call me Janet.
It's all just fun and games until the flying monkeys attack.
mjpb3 is offline   you may: quote & reply,
Post a new thread. Add a reply to this thread. Indicate all threads in this forum as read. Subscribe to this forum. RSS feed: this forum RSS feed: all forums
Go Back   LucasForums > Network > Knights of the Old Republic > Community > The Outlander Club > Jade Empire modding (see first post for info summary)

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -4. The time now is 05:44 AM.

LFNetwork, LLC ©2002-2011 - All rights reserved.
Powered by vBulletin®
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.