Heavy Iron Modding

EvilEngine/HIP (File Format): Difference between revisions

Page Discussion

EvilEngine/HIP (File Format) (view source)

Revision as of 23:13, 29 August 2022

6,737 bytes added ,  1 year ago
Add new info, reorganize things, improve wording
mNo edit summary
(Add new info, reorganize things, improve wording)
Line 1:
'''HIP archives''', also referred to as '''HIP/HOP files''' or '''packages''', are container files used by Heavy Iron Studios into theirstore 5all RenderWarelevel gamesdata coveredin ontheir thisEvil wikiEngine games. They are usedmade toup storeof [[assets]], which are the resources and objects used to create levels, characters and menus for the games.
 
In [[:Category:Scooby|Scooby-Doo! Night of 100 Frights]], all HIP archives use the file extension <code>.HIP</code>. [[:Category:Scooby|SpongeBob SquarePants: Battle for Bikini Bottom]] introduced the file extension <code>.HOP</code>, which has the exact same file format as <code>.HIP</code> but is typically used to store different types of assets. Each level and some global scenes (such as player data) are stored asLocalized <code>.HIP</code>/<code>.HOP</code> pairsfiles were added in BFBB[[:Category:TSSM|The andSpongeBob subsequentSquarePants games,Movie]] and thelater restgames ofto thesupport HIPdifferent archives just uselanguages (<code>_US.HIP</code> (boot.HIP, font<code>_JP.HIP</code>, etc.).
 
==File Format==
There are three known revisions of the HIP archive format: the first one used in [[:Category:Scooby|Scooby-Doo! Night of 100 Frights]], a second one used in [[:Category:BFBB|SpongeBob SquarePants: Battle For Bikini Bottom]] and a third one which is used in [[:Category:TSSM|The SpongeBob SquarePants Movie]], [[:Category:Incredibles|The Incredibles]] and [[:Category:ROTU|The Incredibles: Rise of the Underminer]].
 
===Integers===
A HIP archive is stored as a hierarchical structure of ''blocks''. Each block has a 4-byte ID and length, and can contain data as well as child blocks. The ID is typically human-readable and can be read as a 4-byte ASCII string, although it's technically read as an int in BFBB's source code. The length determines how many bytes the entire block takes up, including both its data and child blocks. The header is stored as follows:
All integers in HIP archives are unsigned 32-bit (4 bytes long). They are always written in [https://en.wikipedia.org/wiki/Endianness big endian], regardless of platform.
 
Integers are referred to as <code>int</code> in format descriptions.
<pre>
 
int blockID
===Strings===
int blockLength
All strings in HIP archives are [https://en.wikipedia.org/wiki/Null-terminated_string null-terminated] (i.e. end with a NULL byte, <code>'\0'</code> in C/C++). They do not have a character count at the beginning. All strings are padded at the end so that their length is a multiple of two bytes, meaning:
</pre>
* Strings with an odd length (<code>"abc"</code>) will end with one NULL (hex <code>61 62 63 00</code>).
* Strings with an even length (<code>"abcd"</code>) with end with two NULLs (hex <code>61 62 63 64 00 00</code>).
* Empty strings are written as hex <code>00 00</code>.
 
Strings are referred to as <code>string</code> in format descriptions.
 
===Structure===
A HIP archive is stored as a tree of ''blocks'' (also known as chunks), where blocks can contain arbitrary data as well as any number of child blocks.
 
Each block begins with an <code>int</code> human-readable ID that determines what kind of data is present in the block, followed by an <code>int</code> length that is the total amount of bytes taken up by the block's data + child blocks, if any. After the ID and length, the block's data is stored; its format depends on the block ID (see [[#Blocks|Blocks]]). Any remaining bytes after the data make up the block's child blocks, which have the same format as normal blocks. Child blocks are parsed repeatedly until the end of the parent block is reached.
Following the header, any block-specific data is stored, followed by any child blocks.
 
===Overall Structure===
The basic block hierarchy common to all HIP archives is as follows:
 
Line 38 ⟶ 46:
** [[#DHDR|DHDR]]
** [[#DPAK|DPAK]]
 
===Assets===
Assets are the most important part of HIP archives; they're like the files in ZIP archives. Assets hold all of the raw data for textures, models, sounds, and even objects within a level. Assets also have a name, as well as a unique ID (which is generated from the name using a [[Data Types#AssetID|hash algorithm]]). The ID is used by the game to find/identify assets. Assets also have a type ID that tells the game how to interpret the asset's raw data. The asset's data is stored at a specified offset in the [[#DPAK|DPAK]] block. HIP archives do not use any compression for asset data.
 
===Layers===
Layers are used by HIP archives to group assets of specific types together. For example, <code>TEXTURE</code> layers contain all of the [[RWTX]] assets in a HIP archive and <code>MODEL</code> layers contain all of the [[MODL]] assets. All assetsAssets in athe HIPsame archivelayer have their raw data groupedstored togethernext byto layereach insideother in the [[#DPAK|DPAK]] block,. withThis paddingis inmainly betweenused eachto asset'soptimize datathe memory layout of assets in-game, as well as enforce the order in betweenwhich eachassets layerare loaded (e.g. [[#DPAK|DPAK]]<code>TEXTURE</code> explainslayers moremust aboutbe howstored paddingbefore and<code>MODEL</code> alignmentin oforder assetsfor andtextures layersto be applied to models worksproperly).
 
The following table lists all possibleknown layer types, and their corresponding type IDs in each game,. andAn whatempty [[Asset#Listcell ofmeans assetthe types|assetlayer types]]type theyis contain:not present in the game.
 
{| class="wikitable"
! Name !! [[:Category:Scooby|Scooby]] !! [[:Category:BFBB|BFBB]] !! [[:Category:TSSM|TSSM]] !! [[:Category:Incredibles|Incredibles]] !! [[:Category:ROTU|ROTU]] !! Asset Types
|-
| DEFAULT || 0 || 0 || 0 || 0 || 0
| DEFAULT || 0 || 0 || 0 || 0 || 0 || [[ALST]], [[ANIM]] (Scooby), [[ATBL]], [[ATKT]], [[BOUL]], [[BUTN]], [[CAM]], [[CCRV]], [[CNTR]], [[COLL]], [[COND]], [[CSNM]], [[DEST]], [[DPAT]], [[DSCO]], [[DSTR]], [[DTRK]], [[DUPC]], [[DYNA]], [[EGEN]], [[ENV]], [[FLY]], [[FOG]], [[GRSM]], [[GRUP]], [[GUST]], [[HANG]], [[JAW]], [[LITE]], [[LKIT]], [[LOBM]], [[LODT]], [[MAPR]], [[MINF]], [[MPHT]], [[MRKR]], [[MVPT]], [[NGMS]], [[NPC]], [[NPCS]], [[ONEL]], [[PARE]], [[PARP]], [[PARS]], [[PEND]], [[PGRS]], [[PICK]], [[PIPT]], [[PKUP]], [[PLAT]], [[PLYR]], [[PORT]], [[PRJT]], [[RANM]], [[RAW]], [[SCRP]], [[SDFX]], [[SFX]], [[SGRP]], [[SHDW]], [[SHRP]], [[SIMP]], [[SLID]], [[SPLN]], [[SPLP]], [[SSET]], [[SUBT]], [[SURF]], [[TEXT]], [[TIMR]], [[TPIK]], [[TRIG]], [[TRWT]], [[UI]], [[UIFT]], [[UIM]], [[VIL]], [[VILP]], [[VOLU]], [[ZLIN]]
|-
| TEXTURE || 1 || 1 || 1 || 1 || 1 || [[RWTX]]
|-
| TEXTURE_STRM || || || 2 || 2 || 2 || [[BINK]], [[TEXS]], [[WIRE]]
|-
| BSP || 2 || 2 || 3 || 3 || 3 || [[BSP]], [[JSP]]
|-
| MODEL || 3 || 3 || 4 || 4 || 4 || [[MODL]]
|-
| ANIMATION || 4 || 4 || 5 || 5 || 5 || [[ANIM]] (BFBB/TSSM/Incredibles/ROTU)
|-
| VRAM || 5 || 5 || 6 || 6 || 6 ||
|-
| SRAM || 6 || 6 || 7 || 7 || 7 || [[CSSS]], [[SND]], [[SNDS]]
|-
| SNDTOC || 7 || 7 || 8 || 8 || 8 || [[CTOC]] (Scooby/BFBB), [[SNDI]]
|-
| CUTSCENE || 8 || 8 || 9 || 9 || 9 || [[CSN]], [[CTOC]] (TSSM/Incredibles/ROTU)
|-
| CUTSCENETOC || 9 || 9 || 10 || 10 || 10 ||
|-
| JSPINFO || || 10 || 11 || 11 || 11 || [[JSP]]
|}
 
===Strings=Layout====
HIP archives often contain multiple layers of the same type, usually placed right next to each other. In the vanilla games, assets are distributed across those layers so that each layer has roughly the same number of assets. For example, if a HIP archive has 3 <code>TEXTURE</code> layers and 30 total [[RWTX]] assets, each <code>TEXTURE</code> layer will contain 10 [[RWTX]] assets. This was most likely done for multi-threading purposes (loading multiple layers in parallel).
All strings in HIP archives are [https://en.wikipedia.org/wiki/Null-terminated_string null-terminated], i.e. end with a 0 byte (<code>\x00</code>). They do not have a character count at the beginning. All strings are padded at the end so that their length is a multiple of two bytes. This means that all strings will have either one or two 0s at the end. Empty strings are simply stored as two 0s.
 
[[:Category:TSSM|The SpongeBob SquarePants Movie]] and later games have JSP layer "groups" to support multiple JSPs in one level. These are made up of 3 <code>BSP</code> layers followed by one <code>JSPINFO</code> layer. JSP layer groups often appear multiple times in a HIP archive. A <code>.HIP</code>/<code>.HOP</code> pair will always have the same number of JSP layer groups, even though they are completely empty in <code>.HIP</code> files.
Strings are referred to as <code>string</code> in the [[#Block|Block]] format descriptions.
 
The following tables specify the layout of layers for each game. Each table is sorted by the layers' order of appearance in HIP archives.
===Endian===
 
HIP archives are stored as [https://en.wikipedia.org/wiki/Endianness big endian] in all versions of each game.
* The '''Name''' column is the internal name for each layer type.
* The '''Count''' column is how many times each layer type repeats.
* The '''Asset Types''' column lists what asset types appear in each layer type and whether they mainly appear in <code>.HIP</code>, <code>.HOP</code>, or localized <code>.HIP</code> files (e.g. <code>_US.HIP</code>).
 
{| class="wikitable"
|+ [[:Category:Scooby|Scooby-Doo! Night of 100 Frights]]
! Name !! Count !! Asset Types
|-
| TEXTURE || 1 || HIP: [[RWTX]]
|-
| BSP || 1 || HIP: [[BSP]]
|-
| MODEL || 1 || HIP: [[MODL]]
|-
| ANIMATION || 1 ||
|-
| DEFAULT || 1 || HIP: [[ALST]], [[ANIM]], [[ATBL]], [[BUTN]], [[CAM]], [[CNTR]], [[COND]], [[CSNM]], [[DPAT]], [[DSTR]], [[EGEN]], [[ENV]], [[FOG]], [[GRUP]], [[GUST]], [[HANG]], [[LITE]], [[LOBM]], [[MAPR]], [[MINF]], [[MPHT]], [[MRKR]], [[MVPT]], [[NPC]], [[PARE]], [[PARS]], [[PEND]], [[PICK]], [[PKUP]], [[PLAT]], [[PLYR]], [[PORT]], [[PRJT]], [[SCRP]], [[SFX]], [[SIMP]], [[SURF]], [[TEXT]], [[TIMR]], [[TRIG]], [[UI]], [[UIFT]], [[VOLU]]
|-
| CUTSCENE || 1 || HIP: [[CSN]]
|-
| SRAM || 1 || HIP: [[SND]], [[SNDS]]
|-
| SNDTOC || 1 || HIP: [[CTOC]], [[SNDI]]
|}
 
{| class="wikitable"
|+ [[:Category:BFBB|SpongeBob SquarePants: Battle for Bikini Bottom]]
! Name !! Count !! Asset Types
|-
| TEXTURE || 3 || HOP: [[RWTX]]
|-
| BSP || 3 || HOP: [[JSP]] (RpClump)
|-
| JSPINFO || 1 || HOP: [[JSP]] (JSP Info)
|-
| MODEL || 3 || HOP: [[MODL]]
|-
| ANIMATION || 1 || HOP: [[ANIM]]
|-
| DEFAULT || 1 || HIP: [[ALST]], [[ATBL]], [[BOUL]], [[BUTN]], [[CAM]], [[CNTR]], [[COND]], [[CRDT]], [[CSNM]], [[DPAT]], [[DSCO]], [[DSTR]], [[DYNA]], [[EGEN]], [[ENV]], [[FLY]], [[FOG]], [[GRUP]], [[LKIT]], [[MINF]], [[MRKR]], [[MVPT]], [[PARE]], [[PARP]], [[PARS]], [[PICK]], [[PKUP]], [[PLAT]], [[PLYR]], [[PORT]], [[RAW]], [[SFX]], [[SHRP]], [[SIMP]], [[SURF]], [[TEXT]], [[TIMR]], [[TRIG]], [[UI]], [[UIFT]], [[VIL]], [[VILP]]
 
HOP: [[COLL]], [[JAW]], [[LODT]], [[MAPR]], [[PIPT]], [[SHDW]]
|-
| CUTSCENE || 1 || HOP: [[CSN]]
|-
| SRAM || 1 || HOP: [[SND]], [[SNDS]]
|-
| SNDTOC || 1 || HOP: [[CTOC]], [[SNDI]]
|}
 
{| class="wikitable"
! Name !! Count !! Asset Types
|+ [[:Category:TSSM|The SpongeBob SquarePants Movie]]
|-
| TEXTURE || 3 || HOP: [[RWTX]]
|-
| TEXTURE_STRM || 1 || HIP: [[TEXS]]
|-
| BSP || 3 || HOP: [[JSP]] (RpClump)
|-
| JSPINFO || 1 || HOP: [[JSP]] (JSP Info)
|-
| MODEL || 3 || HOP: [[MODL]]
|-
| ANIMATION || 1 ||
|-
| DEFAULT || 1 || HIP: [[ALST]], [[ATBL]], [[BOUL]], [[BUTN]], [[CAM]], [[CNTR]], [[COND]], [[CSNM]], [[DPAT]], [[DSCO]], [[DYNA]], [[EGEN]], [[ENV]], [[FLY]], [[FOG]], [[GRUP]], [[LKIT]], [[MINF]], [[MRKR]], [[MVPT]], [[PARE]], [[PARP]], [[PARS]], [[PICK]], [[PKUP]], [[PLAT]], [[PLYR]], [[PORT]], [[RANM]], [[RAW]], [[SCRP]], [[SDFX]], [[SGRP]], [[SIMP]], [[SPLN]], [[SURF]], [[TIMR]], [[TRIG]], [[UIM]]
 
HOP: [[ANIM]], [[COLL]], [[DEST]], [[JAW]], [[LODT]], [[MAPR]], [[PIPT]], [[SHRP]]
 
Localized HIP: [[CRDT]], [[TEXT]]
|-
| CUTSCENE || 1 || HOP: [[CSN]]
|-
| SRAM || 1 || HOP: [[CSSS]], [[SND]], [[SNDS]]
|-
| SNDTOC || 1 || HOP: [[SNDI]]
|-
| CUTSCENETOC || 1 || HOP: [[CTOC]]
|}
 
{| class="wikitable"
|+ [[:Category:Incredibles|The Incredibles]]
! Name !! Count !! Asset Types
|-
| TEXTURE || 3 || HOP: [[RWTX]]
|-
| TEXTURE_STRM || 1 || HIP: [[BINK]], [[TEXS]]
|-
| BSP || 3 || HOP: [[JSP]] (RpClump)
|-
| JSPINFO || 1 || HOP: [[JSP]] (JSP Info)
|-
| MODEL || 3 || HOP: [[MODL]]
|-
| ANIMATION || 1 ||
|-
| DEFAULT || 1 || HIP: [[ALST]], [[ATBL]], [[ATKT]], [[BOUL]], [[BUTN]], [[CAM]], [[CNTR]], [[COND]], [[CSNM]], [[DPAT]], [[DUPC]], [[DYNA]], [[ENV]], [[FOG]], [[GRUP]], [[LKIT]], [[MINF]], [[MRKR]], [[MVPT]], [[NPCS]], [[ONEL]], [[PGRS]], [[PICK]], [[PLAT]], [[PLYR]], [[PORT]], [[RANM]], [[RAW]], [[SCRP]], [[SDFX]], [[SGRP]], [[SIMP]], [[SLID]], [[SPLP]], [[SSET]], [[SURF]], [[TIMR]], [[TPIK]], [[TRIG]], [[TRWT]], [[UIM]], [[VIL]], [[WIRE]], [[ZLIN]]
 
HOP: [[ANIM]], [[COLL]], [[DEST]], [[DTRK]], [[GRSM]], [[LODT]], [[MAPR]], [[NGMS]], [[PIPT]], [[SHRP]], [[SPLN]]
 
Localized HIP: [[CRDT]], [[SUBT]], [[TEXT]]
|-
| CUTSCENE || 1 || HOP: [[CSN]]
|-
| SRAM || 1 || HOP: [[CSSS]], [[SND]], [[SNDS]]
 
Localized HIP: [[SNDS]]
|-
| SNDTOC || 1 || HOP: [[SNDI]]
 
Localized HIP: [[SNDI]]
|-
| CUTSCENETOC || 1 || HOP: [[CTOC]]
|}
 
{| class="wikitable"
|+ [[:Category:ROTU|The Incredibles: Rise of the Underminer]]
! Name !! Count !! Asset Types
|-
| TEXTURE || 3 || HOP: [[RWTX]]
|-
| TEXTURE_STRM || 1 || HIP: [[TEXS]]
 
HOP: [[BINK]]
|-
| BSP || 3 || HOP: [[JSP]] (RpClump)
|-
| JSPINFO || 1 || HOP: [[JSP]] (JSP Info)
|-
| MODEL || 3 || HOP: [[MODL]]
|-
| ANIMATION || 1 || HOP: [[ANIM]]
|-
| DEFAULT || 1 || HIP: [[ALST]], [[ATBL]], [[BOUL]], [[CAM]], [[CCRV]], [[CNTR]], [[COND]], [[DPAT]], [[DYNA]], [[ENV]], [[FOG]], [[GRUP]], [[LKIT]], [[MINF]], [[MRKR]], [[MVPT]], [[NPCS]], [[PARE]], [[PARP]], [[PARS]], [[PGRS]], [[PICK]], [[PLAT]], [[PLYR]], [[PORT]], [[RAW]], [[SCRP]], [[SDFX]], [[SGRP]], [[SIMP]], [[SPLN]], [[SURF]], [[TIMR]], [[TPIK]], [[TRIG]], [[TRWT]], [[UIM]], [[VOLU]], [[WIRE]]
 
HOP: [[COLL]], [[DEST]], [[JAW]], [[LODT]], [[MAPR]], [[NGMS]], [[PIPT]], [[SHRP]], [[SPLN]]
 
Localized HIP: [[CRDT]], [[SUBT]], [[TEXT]]
|-
| CUTSCENE || 1 ||
|-
| SRAM || 1 || HOP: [[SND]], [[SNDS]]
 
Localized HIP: [[SNDS]]
|-
| SNDTOC || 1 || HOP: [[SNDI]]
 
Localized HIP: [[SNDI]]
|-
| CUTSCENETOC || 1 ||
|}
 
==Blocks==
This section contains the format for eachall known block describedtypes in [[#OverallHIP Structure|Overall Structure]]archives. Each section heading refers tois the block's 4-bytetype ID as if it was a human-readable string.
 
===HIPA===
Line 262 ⟶ 425:
'''alignment''' is the multiple of bytes that the asset's data aligns to. This value can be -1 (or any negative value), which means it uses the "default" alignment value for the asset's specific type ID. The default alignment value for each asset type can be found in <code>.LIP</code>/<code>.LOP</code> files in BFBB and Scooby (either 16 or 32 for most types).
 
'''name''' is the asset's name, which also determines the asset's ID. This was (unfortunately) trimmed to 31 characters (plus the NULL byte) when stored in vanilla HIP files, despite HIP files supporting longer strings, however, the asset ID was calculated from the asset's original name. This means that there are sometimes multiple assets in a HIP file with the same name but different IDs.
'''name''' is the asset's name, which also determines the asset's ID.
 
'''filename''' is the asset's source filename if the <code>SOURCE_FILE</code> flag is set in the parent [[#AHDR|AHDR]] block.
Retrieved from "https://heavyironmodding.org/wiki/Special:MobileDiff/6163"