1,536
edits
m (5 revisions imported) |
(Rewrite/reorganize HIP format spec) |
||
Line 1:
'''HIP archives''', also referred to as '''HIP/HOP files''' or '''packages''', are container files used by Heavy Iron Studios in their 5 RenderWare games covered on this wiki. They are used to store [[assets]], which are the resources and objects used to create levels, characters and menus for the games.
In [[:Category:Scooby|Scooby-Doo! Night of 100 Frights]], all HIP archives use the file extension <code>.HIP</code>. [[:Category:Scooby|SpongeBob SquarePants: Battle for Bikini Bottom]] introduced the file extension <code>.HOP</code>, which has the exact same file format as <code>.HIP</code> but is typically used to store different types of assets. Each level and some global scenes (such as player data) are stored as <code>.HIP</code>/<code>.HOP</code> pairs in BFBB and subsequent games, and the rest of the HIP archives just use <code>.HIP</code> (boot.HIP, font.HIP, etc.).
==
There are three known revisions of the HIP archive format: the first one used in [[:Category:Scooby|Scooby-Doo! Night of 100 Frights]], a second one used in [[:Category:BFBB|SpongeBob SquarePants: Battle For Bikini Bottom]] and a third one which is used in [[:Category:TSSM|The SpongeBob SquarePants Movie]], [[:Category:Incredibles|The Incredibles]] and [[:Category:ROTU|The Incredibles: Rise of the Underminer]].
A HIP archive is stored as a hierarchical structure of ''blocks''. Each block has a 4-byte ID and length, and can contain data as well as child blocks. The ID is typically human-readable and can be read as a 4-byte ASCII string, although it's technically read as an int in BFBB's source code. The length determines how many bytes the entire block takes up, including both its data and child blocks. The header is stored as follows:
<pre>
int blockID
int
</pre>
Following the header, any block-specific data is stored, followed by any child blocks.
===
The basic block hierarchy common to all HIP archives is as follows:
* [[#HIPA|HIPA]]
* [[#PACK|PACK]]
** [[#PVER|PVER]]
** [[#PFLG|PFLG]]
** [[#PCNT|PCNT]]
** [[#PCRT|PCRT]]
** [[#PMOD|PMOD]]
** [[#PLAT|PLAT]] (Not present in Scooby)
* [[#DICT|DICT]]
** [[#ATOC|ATOC]]
*** [[#AINF|AINF]]
*** [[#AHDR|AHDR]]+ (Multiple can be present)
**** [[#ADBG|ADBG]]
** [[#LTOC|LTOC]]
*** [[#LINF|LINF]]
*** [[#LHDR|LHDR]]+ (Multiple can be present)
**** [[#LDBG|LDBG]]
* [[#STRM|STRM]]
** [[#DHDR|DHDR]]
** [[#DPAK|DPAK]]
===Layers===
Layers are used by HIP archives to group assets of specific types together. For example, <code>TEXTURE</code> layers contain all of the [[RWTX]] assets in a HIP archive and <code>MODEL</code> layers contain all of the [[MODL]] assets. All assets in a HIP archive have their data grouped together by layer inside the [[#DPAK|DPAK]] block, with padding in between each asset's data as well as in between each layer. [[#DPAK|DPAK]] explains more about how padding and alignment of assets and layers works.
TODO: list all known type IDs
===Strings===
All strings in HIP archives are [https://en.wikipedia.org/wiki/Null-terminated_string null-terminated], i.e. end with a 0 byte (<code>\x00</code>). They do not have a character count at the beginning. All strings are padded at the end so that their length is a multiple of two bytes. This means that all strings will have either one or two 0s at the end. Empty strings are simply stored as two 0s.
Strings are referred to as <code>string</code> in the [[#Block|Block]] format descriptions.
===Endian===
HIP archives are stored as [https://en.wikipedia.org/wiki/Endianness big endian] in all versions of each game.
==Blocks==
This section contains the format for each block described in [[#Overall Structure|Overall Structure]]. Each section heading refers to the block's 4-byte ID.
===HIPA===
This block contains no data or child blocks. It is used to mark the start of the file, and the ID can serve as a [https://en.wikipedia.org/wiki/Magic_number_(programming)#In_files magic number] to identify the file's type.
===PACK===
This
===PVER===
This
<pre>
int subVersion
Line 47 ⟶ 68:
int compatible
</pre>
TODO: document all possible values
===PFLG===
This
<pre>
int flags
</pre>
* In all games, this value always includes 0x2E in the flags. Its meaning is unknown.
* In BFBB, every file except font2.HIP has some extra flags:
** 0x290000 is used in GC US hips
** 0x2A0000 is used in Xbox hips
** 0x2C0000 is used in PS2 hips
** 0x510000 is used in GC mn-pal/mnu3.HIP and mn-pal/mnu3.HOP
** 0x02000000 is used in all BFBB hips except font2.HIP
===PCNT===
This block contains counts of specific things, only the first 2 are used by the game.
<pre>
int assetCount
int layerCount
int maxAssetSize
int maxLayerSize
int maxXformAssetSize
</pre>
'''assetCount''' determines how many assets are in the HIP archive and thus how many [[#AHDR|AHDR]] blocks should be present.
'''layerCount''' determines how many layers are in the HIP archive and thus how many [[#LHDR|LHDR]] blocks should be present.
'''maxAssetSize''' is the largest asset size in bytes out of all assets in the archive.
'''maxLayerSize''' is the size in bytes of the layer that takes up the most space in the [[#DPAK|DPAK]] block, minus the padding bytes at the end.
'''maxXformAssetSize''' is the largest asset size in bytes out of all assets specifically with the <code>READ_TRANSFORM</code> flag set (see [[#AHDR|AHDR]] flags).
===PCRT===
This
<pre>
time_t createdDate
string createdDateString
</pre>
[https://en.cppreference.com/w/c/chrono/time_t <code>time_t</code>] is stored as a 4-byte int and originates from C/C++. In HIP archives, it is the number of seconds since 00:00, Jan 1 1970 UTC, with a timezone offset of UTC-7:00 (Pacific Time).
'''createdDateString''' always ends with a newline character (<code>'\n'</code>, <code>'\x0A'</code>) in Scooby.
===PMOD===
This block contains the latest modification date of the archive.
<pre>
time_t modifiedDate
</pre>
See [[#PCRT|PCRT]] for a definition of <code>time_t</code>.
===PLAT===
This
BFBB:
<pre>
int platformID
string
string
string language
string
</pre>
TSSM, Incredibles, ROTU:
<pre>
int platformID
string language
string
string
</pre>
'''platformID''' is a 4-byte ID referring to which platform the archive was built for.
'''platformName''' is a human-readable string corresponding to the platformID.
'''region''' is the archive's target region, either NTSC or PAL.
'''language''' is the archive's target language.
'''gameName''' is the archive's target game. This is always "Sponge Bob" in BFBB and "Incredibles" in TSSM, Incredibles, and ROTU.
TODO: list all possible values for everything
===DICT===
This
===ATOC===
This section holds all the asset entries for the HIP archive. It contains no data and a [[#AINF|AINF]] child block followed by a variable number of [[#AHDR|AHDR]] child blocks. The number of AHDR blocks should match the '''assetCount''' value found in [[#PCNT|PCNT]]. All AHDR blocks together form a list and is sorted by the '''id''' value found within each block when building the HIP archive.
===AINF===
This block is unused by the game.
<pre>
int
</pre>
This value is always 0.
===AHDR===
This block defines an entry for an [[asset]]. It contains some data followed by an [[#ADBG|ADBG]] child block.
<pre>
int id
int type
int offset
int
int plus
int flags
</pre>
'''id''' is the asset's unique 4-byte ID, which is calculated by the asset's name using a variation of the BKDR hash algorithm, described [[Data Types#AssetID|here]].
'''type''' is the asset's 4-byte type ID, which is typically human-readable and can be read as an ASCII string if preferred. All known type IDs can be found [[Asset#List of asset types|here]].
'''offset''' is the starting position of the asset's data, relative to the beginning of the file.
'''size''' is the length in bytes of the asset's data.
'''plus''' is the amount of padding bytes present between the end of the asset's data and the next asset in the layer's data. It's calculated from the '''alignment''' value in the [[#ADBG|ADBG]] block. The last asset in each layer has a plus value of 0.
'''flags''' is a bitfield of settings which specifies how the asset's data is stored and how it should be handled by the game:
* 0x1 - SOURCE_FILE - The asset's data was sourced from an external file. The filename/path can be found in the [[#ADBG|ADBG]] child block.
* 0x2 - SOURCE_VIRTUAL - The asset's data was created within the level editor Heavy Iron used. The filename in [[#ADBG|ADBG]] is empty.
* 0x4 - READ_TRANSFORM - The asset's data is stored in a special format which needs to be "transformed" by the game into a runtime-specific format. This applies to all [[RenderWare]] assets, such as [[MODL]], which is stored as a <code>.dff</code> file and needs to be transformed into an "RpClump" (RenderWare object) at runtime.
* 0x8 - WRITE_TRANSFORM - The asset's data needs to be transformed from a runtime-specific format into a special binary format, likely used by Heavy Iron's level editor.
===ADBG===
This block defines debug-specific info for its parent [[#AHDR|AHDR]] block. None of this is used by the release builds of any games, but Heavy Iron left it in anyways, and thank goodness for that.
<pre>
int alignment
string
string
</pre>
'''alignment''' is the multiple of bytes that the asset's data aligns to. This value can be -1 (or any negative value), which means it uses the "default" alignment value for the asset's specific type ID. The default alignment value for each asset type can be found in <code>.LIP</code>/<code>.LOP</code> files in BFBB and Scooby (either 16 or 32 for most types).
'''name''' is the asset's name, which also determines the asset's ID.
'''filename''' is the asset's source filename if the <code>SOURCE_FILE</code> flag is set in the parent [[#AHDR|AHDR]] block.
'''checksum''' is the checksum of the asset's data. The algorithm to calculate the checksum is currently unknown, but it's unused by the game anyways.
===LTOC===
This section holds all the layer entries for the HIP archive. It contains no data and a [[#LINF|LINF]] child block followed by a variable number of [[#LHDR|LHDR]] child blocks. The number of LHDR blocks should match the '''layerCount''' value found in [[#PCNT|PCNT]].
===LINF===
This block is unused by the game.
<pre>
int
</pre>
This value is always 0.
===LHDR===
This block defines an entry for a layer. It contains some data followed by an [[#LDBG|LDBG]] child block.
<pre>
int
int
int[
</pre>
'''type''' is the layer's type ID, which determines what types of assets are stored within this layer. Multiple layers can have the same type ID, which means that assets will be divided evenly between those layers. [[#Layers|Layers]] lists all known type IDs.
'''assetCount''' is the number of assets in this layer.
'''assetIDs''' is a list of all the assets' IDs. The order of IDs in this list determines the order that the assets' data will be stored in [[#DPAK|DPAK]].
===LDBG===
This block is unused by the game.
<pre>
int
</pre>
This value is always -1 (0xFFFFFFFF).
===STRM===
This
===DHDR===
This block is unused by the game.
<pre>
int
</pre>
This value is always -1 (0xFFFFFFFF).
===DPAK===
This
Each layer is aligned to a fixed size which depends on the target platform:
* GameCube: 32 bytes
* PS2/Xbox: 2048 bytes
Each asset is aligned to the alignment value specified by their corresponding [[#ADBG|ADBG]] block.
In order to align assets and layers, padding bytes are used. All padding bytes have the value <code>0x33</code>.
* All assets, ''except'' for the last asset in each layer, have padding bytes after them. The plus value in each asset's [[#AHDR|AHDR]] block is set to the amount of padding bytes used.
* All layers, including the last one, have padding at the end. The '''maxLayerSize''' value in [[#PCNT|PCNT]] does not include the amount of padding bytes used.
If there are 0 assets ('''assetCount''' in [[#PCNT|PCNT]] is 0), this block contains no data. Otherwise:
<pre>
int paddingAmount
char[paddingAmount] padding
char[] data
</pre>
There is padding between the start of the block and the beginning of the asset data. The amount of padding is calculated from the fixed layer alignment size (described above), minus the first 4 bytes which is reserved for '''paddingAmount'''. It specifies how many padding bytes will follow.
==Tools==
[[HipHopFile]] is a library which can be used to work with HIP files.
==External links==
* [https://github.com/bfbbdecomp/bfbb/blob/master/src/Core/x/xhipio.cpp xhipio.cpp] - BFBB's source code for reading basic blocks and values from HIP archives
* [https://github.com/bfbbdecomp/bfbb/blob/master/src/Core/x/xpkrsvc.cpp xpkrsvc.cpp] - BFBB's source code for reading all assets and layers from BFBB-specific HIP archives
[[Category:File Format]]
|