Catalog.json: Difference between revisions

No edit summary
m Use proper Wikipedia interwiki suffix
 
(17 intermediate revisions by one other user not shown)
Line 3: Line 3:
'''catalog.json''' is a [[Wikipedia:JSON|JSON]] file used by the [[com.unity.addressables]] package in the [[Unity]] game engine. It is a list of all assets used in a game.
'''catalog.json''' is a [[Wikipedia:JSON|JSON]] file used by the [[com.unity.addressables]] package in the [[Unity]] game engine. It is a list of all assets used in a game.


The name of the file can be different in each game. Most often it is <code>catalog_data.json</code>, but other options like <code>catalog_catalog.json</code> have also been found. The suffix (like <code>_data</code> or <code>_catalog</code>) can be chosen by the game developer, and some games can even have multiple content catalogs with different suffixes.
__TOC__
 
The name of the file can be different in each game. Most often it is <code>catalog_data.json</code>, but other options like <code>catalog_catalog.json</code> have also been found. The suffix (like <code>data</code> or <code>catalog</code>) can be chosen by the game developer, and some games can even have multiple content catalogs with different suffixes.


In [[Unity]], the contents of the <code>catalog.json</code> file can be accessed through the [https://docs.unity3d.com/Packages/com.unity.addressables@1.19/api/UnityEngine.AddressableAssets.ResourceLocators.ContentCatalogData.html ContentCatalogData] class.
In [[Unity]], the contents of the <code>catalog.json</code> file can be accessed through the [https://docs.unity3d.com/Packages/com.unity.addressables@1.19/api/UnityEngine.AddressableAssets.ResourceLocators.ContentCatalogData.html ContentCatalogData] class.
Line 11: Line 13:
{{DataStructure|
{{DataStructure|
   {{DataStructureField|m_LocatorId|string|Name of this catalog, for example: <code>AddressablesMainContentCatalog</code>}}
   {{DataStructureField|m_LocatorId|string|Name of this catalog, for example: <code>AddressablesMainContentCatalog</code>}}
   {{DataStructureField|m_BuildResultHash|string|Hash of the catalog build, for example: <code>71d0441090abbc8f1a7305f2ea3c8b62</code>. Notably, it seems to be different from [[catalog.hash]]}}
   {{DataStructureField|m_BuildResultHash|string|Hash of the catalog build, for example: <code>71d0441090abbc8f1a7305f2ea3c8b62</code>. Notably, it seems to be different from [[catalog.hash]].}}
   {{DataStructureField|m_InstanceProviderData|provider object|[[InstanceProvider]] information. See [[#Provider object structure]] for details.}}
   {{DataStructureField|m_InstanceProviderData|provider object|[[InstanceProvider]] information. See [[#Provider object structure]] for details.}}
   {{DataStructureField|m_SceneProviderData|provider object|[[SceneProvider]] information. See [[#Provider object structure]] for details.}}
   {{DataStructureField|m_SceneProviderData|provider object|[[SceneProvider]] information. See [[#Provider object structure]] for details.}}
Line 23: Line 25:
     {{DataStructureField||string|An internal ID. May be a weird path, for example: <code>{UnityEngine.AddressableAssets.Addressables.RuntimePath}/Android/crilocaldata_assets_all.bundle</code>; a 32-character-long hexadecimal string, for example: <code>033f3660efceadd40b8cd2845b485652</code>; an asset path, for example: <code>Assets/CriData/Addressables/dice20.daily.0.acb{{=}}4493e0.asset</code>; a short name, for example: <code>EfficientBlur</code>; a {{Tooltip|URL|The URL does not have to be reachable; it is not used for downloading assets, it is only used as an ID.}}, for example: <code><nowiki>http://rizastcdn.pigeongames.cn/default/Android/1a2a40bd550976c58c842045215f204e.bundle</nowiki></code>; and it probably can be any other UTF-8 string.}}
     {{DataStructureField||string|An internal ID. May be a weird path, for example: <code>{UnityEngine.AddressableAssets.Addressables.RuntimePath}/Android/crilocaldata_assets_all.bundle</code>; a 32-character-long hexadecimal string, for example: <code>033f3660efceadd40b8cd2845b485652</code>; an asset path, for example: <code>Assets/CriData/Addressables/dice20.daily.0.acb{{=}}4493e0.asset</code>; a short name, for example: <code>EfficientBlur</code>; a {{Tooltip|URL|The URL does not have to be reachable; it is not used for downloading assets, it is only used as an ID.}}, for example: <code><nowiki>http://rizastcdn.pigeongames.cn/default/Android/1a2a40bd550976c58c842045215f204e.bundle</nowiki></code>; and it probably can be any other UTF-8 string.}}
   }}
   }}
   {{DataStructureField|m_KeyDataString|string|[[Base64]] encoded string containing binary data. See [[#Key data binary format]] for details.}}
   {{DataStructureField|m_KeyDataString|string|[[wikipedia:Base64|Base64]] encoded string containing binary data. See [[#Key data binary format]] for details.}}
   {{DataStructureField|m_BucketDataString|string|[[Base64]] encoded string containing binary data. See [[#Bucket data binary format]] for details.}}
   {{DataStructureField|m_BucketDataString|string|[[wikipedia:Base64|Base64]] encoded string containing binary data. See [[#Bucket data binary format]] for details.}}
   {{DataStructureField|m_EntryDataString|string|[[Base64]] encoded string containing binary data. See [[#Entry data binary format]] for details.}}
   {{DataStructureField|m_EntryDataString|string|[[wikipedia:Base64|Base64]] encoded string containing binary data. See [[#Entry data binary format]] for details.}}
   {{DataStructureField|m_ExtraDataString|string|[[Base64]] encoded string containing binary data. See [[#Extra data binary format]] for details.}}
   {{DataStructureField|m_ExtraDataString|string|[[wikipedia:Base64|Base64]] encoded string containing binary data. See [[#Extra data binary format]] for details.}}
   {{DataStructureField|m_resourceTypes|array|A list of all of the types of assets in the catalog.|
   {{DataStructureField|m_resourceTypes|array|A list of all of the types of assets in the catalog. (This field ''is'' lowercase.)|
     {{DataStructureField||object type object|An object type, for example: <code>{"m_AssemblyName": "UnityEngine.CoreModule, Version{{=}}0.0.0.0, Culture{{=}}neutral, PublicKeyToken{{=}}null", "m_ClassName": "UnityEngine.TextAsset"}</code>. See [[#Object type object structure]] for details.}}
     {{DataStructureField||object type object|An object type, for example: <code>{"m_AssemblyName": "UnityEngine.CoreModule, Version{{=}}0.0.0.0, Culture{{=}}neutral, PublicKeyToken{{=}}null", "m_ClassName": "UnityEngine.TextAsset"}</code>. See [[#Object type object structure]] for details.}}
   }}
   }}
  {{DataStructureField|m_InternalIdPrefixes|array|Unknown. The array was found empty.}}
}}
}}


Line 49: Line 52:
== Binary formats ==
== Binary formats ==


Note: All values are in [[little-endian]].
''Note:'' All values are in [[Wikipedia:little endian|little endian]].


=== Key data binary format ===
=== Key data binary format ===
''Keys'' are mostly string values that identify a ''bucket'' uniquely. They are only used by ''buckets'', and are not used by any other part of the format. Compared to UTF-8 and UTF-16 keys, very few keys actually use numeric values. For example, in ''[[Rizline:Rizline|Rizline]]'', as of [[Rizline:Asset Version 20.1.2.7.1fd57802dc|Asset Version 20.1.2.7.1fd57802dc]], the <code>catalog_catalog.json</code> file contains 2879 keys; out of them, 2828 keys are in UTF-8, 42 keys are UTF-16 (these are keys that include non-[[Wikipedia:ASCII|ASCII]] characters), and only 9 keys are numeric.


{{DataStructure|
{{DataStructure|
Line 57: Line 62:
   {{DataStructureField|keys|array|A list of keys. The length of the list can be determined by either using the <code>key_count</code> field, or reading until the end of the binary data.|
   {{DataStructureField|keys|array|A list of keys. The length of the list can be determined by either using the <code>key_count</code> field, or reading until the end of the binary data.|
     {{DataStructureField||struct|A key. The size of the struct is variable.|
     {{DataStructureField||struct|A key. The size of the struct is variable.|
       {{DataStructureField|key_type|uint8|1-byte number, indicating the type of the key. Known key types are: <code>0</code> - UTF-8 key; <code>1</code> - UTF-16 key; <code>4</code> - int32 key.}}
       {{DataStructureField|key_type|uint8|1-byte number, indicating the type of the key. Known key types are: <code>0</code> - [[Wikipedia:UTF-8|UTF-8]] key; <code>1</code> - [[Wikipedia:UTF-16|UTF-16]] key; <code>4</code> - int32 key.}}
       {{DataStructureField|key_content|union||
       {{DataStructureField|key_content|union||
         {{DataStructureField|text_content|sized string struct|Only present if key type is <code>0</code> or <code>1</code>. The type of the string is UTF-8 if <code>key_type</code> is <code>0</code> and UTF-16 (little-endian) if <code>key_type</code> is <code>1</code>. This field is the key's name. See [[#Sized string binary format]] for details.}}
         {{DataStructureField|text_content|sized string struct|Only present if key type is <code>0</code> or <code>1</code>. The type of the string is UTF-8 if <code>key_type</code> is <code>0</code> and UTF-16 ([[wikipedia:little endian|little endian]]) if <code>key_type</code> is <code>1</code>. This field is the key's name. See [[#Sized string binary format]] for details.}}
         {{DataStructureField|numeric_content|uint32|Only present if key type is <code>4</code>. This field is the key's name.}}
         {{DataStructureField|numeric_content|uint32|Only present if key type is <code>4</code>. This field is the key's name.}}
       }}
       }}
Line 67: Line 72:


=== Bucket data binary format ===
=== Bucket data binary format ===
A ''bucket'' can be thought of as an arbitrary list of ''addressable entries'' with a name (<code>key</code>) attached to it. What the list actually means must be determined separately.


<!-- fun fact, this format consists entirely out of uint32 values. fun! -->
<!-- fun fact, this format consists entirely out of uint32 values. fun! -->
Line 73: Line 80:
   {{DataStructureField|buckets|array|A list of buckets. The length of the list can be determined by either using the <code>bucket_count</code> field, or reading until the end of the binary data.|
   {{DataStructureField|buckets|array|A list of buckets. The length of the list can be determined by either using the <code>bucket_count</code> field, or reading until the end of the binary data.|
     {{DataStructureField||struct|A bucket. The size of the struct is variable.|
     {{DataStructureField||struct|A bucket. The size of the struct is variable.|
       {{DataStructureField|key_offset|uint32|4-byte integer, offset into the <code>KeyData</code> binary blob. Points to the beginning of a key entry associated with this bucket.}}
       {{DataStructureField|key_offset|uint32|4-byte integer, offset into the <code>KeyData</code> binary blob. Points to the beginning of a key entry associated with this bucket. The key can be thought of as being owned by this bucket, as no other bucket owns the same key, and the keys in the <code>KeyData</code> blob are not being pointed to by any other part of the system.}}
       {{DataStructureField|entry_count|uint32|4-byte integer, determining the amount of entries in the <code>entries</code> array.}}
       {{DataStructureField|entry_count|uint32|4-byte integer, determining the amount of entries in the <code>entries</code> array.}}
       {{DataStructureField|entries|array|Array of 4-byte indices. The length of this array is determined by the <code>entry_count</code> field.|
       {{DataStructureField|entry_indices|array|Array of 4-byte indices. The length of this array is determined by the <code>entry_count</code> field.|
         {{DataStructureField||uint32|An index. This value is used to index the array that is contained within the binary blob encoded in <code>m_EntryDataString</code>.}}
         {{DataStructureField||uint32|An index. This value is used to index the array that is contained within the binary blob encoded in <code>m_EntryDataString</code>. The index points to an Entry, that is in this bucket.}}
       }}
       }}
     }}
     }}
Line 86: Line 93:
<!-- fun fact, this format also consists entirely out of uint32 values. fun! -->
<!-- fun fact, this format also consists entirely out of uint32 values. fun! -->
{{DataStructure|
{{DataStructure|
   {{DataStructureField|entry_count|uint32|4-byte integer, indicating the amount of entries stored in the whole structure.}}
   {{DataStructureField|addressable_entry_count|uint32|4-byte integer, indicating the amount of entries stored in the whole structure.}}
   {{DataStructureField|entries|array|A list of entries. The length of the list can be determined by either using the <code>entry_count</code> field, or reading until the end of the binary data.|
   {{DataStructureField|addressable_entries|array|A list of addressable entries. The length of the list can be determined by either using the <code>addressable_entry_count</code> field, or reading until the end of the binary data.|
     {{DataStructureField||struct|An entry. The size of one entry is always 28 bytes (7 * 4 bytes).|
     {{DataStructureField||struct|An addressable entry. The size of one entry is always 28 bytes (7 * 4 bytes).|
       {{DataStructureField|internal_id_index|uint32|4-byte integer, index into the <code>m_InternalIds</code> array inside the parent [[catalog.json]] file.}}
       {{DataStructureField|internal_id_index|uint32|4-byte integer, index into the <code>m_InternalIds</code> array inside the parent [[catalog.json]] file.}}
       {{DataStructureField|resource_provider_index|uint32|4-byte integer, index into the <code>m_ProviderIds</code> array inside the parent [[catalog.json]] file.}}
       {{DataStructureField|resource_provider_index|uint32|4-byte integer, index into the <code>m_ProviderIds</code> array inside the parent [[catalog.json]] file.}}
       {{DataStructureField|unknown_bucket_index|uint32|4-byte integer, index into the array that is contained within the binary blob encoded in <code>m_BucketDataString</code>. {{verify|What is this value's use?}}}}
       {{DataStructureField|dependencies_bucket_index|uint32|4-byte integer, index into the array that is contained within the binary blob encoded in <code>m_BucketDataString</code>. The referenced bucket contains this entry's dependencies - indices of other addressable entries, that need to be downloaded for this one to work properly.}}
       {{DataStructureField|unknown_asset_value|uint32|4-byte integer. Set to 0 for most assets, but set to a seemingly random value from 0 to 4294967295 if the entry's internal ID is either: (1) a 32-character-long hexadecimal string, or (2) a file path ending in <code>.asset</code>. {{verify|What is this value's use?}}}}
       {{DataStructureField|bundled_asset_provider_crc|uint32|4-byte integer. This field is used only if the resource provider for this asset is <code>UnityEngine.ResourceManagement.ResourceProviders.BundledAssetProvider</code>. If that is the case, this field's value is seemingly random, ranging from 0 to 4294967295, most likely being some kind of CRC check. If the resource provider is different, this field's value is 0. {{verify}}}}
       {{DataStructureField|extra_data_block_offset|uint32|4-byte integer, offset into the <code>ExtraData</code> binary blob. Points to the beginning of an extra data entry associated with this entry.}}
       {{DataStructureField|extra_data_block_offset|uint32|4-byte integer, offset into the <code>ExtraData</code> binary blob. Points to the beginning of an extra data entry associated with this entry.}}
       {{DataStructureField|unknown2_bucket_index|uint32|4-byte integer, index into the array that is contained within the binary blob encoded in <code>m_BucketDataString</code>. {{verify|What is this value's use?}}}}
       {{DataStructureField|siblings_bucket_index|uint32|4-byte integer, index into the array that is contained within the binary blob encoded in <code>m_BucketDataString</code>. The referenced bucket contains the family of this addressable entry - the key is like a name for the content of this entry, and other addressable entries that are in the bucket are the "siblings". Siblings ultimately point to the same asset, but have different resource types. For example, an image asset might have two family members: a <code>UnityEngine.Texture2D</code> entry, and a <code>UnityEngine.Sprite</code> entry.}}
       {{DataStructureField|resource_type_index|uint32|4-byte integer, index into the <code>m_resourceTypes</code> array inside the parent [[catalog.json]] file.}}
       {{DataStructureField|resource_type_index|uint32|4-byte integer, index into the <code>m_resourceTypes</code> array inside the parent [[catalog.json]] file.}}
     }}
     }}
Line 101: Line 108:


=== Extra data binary format ===
=== Extra data binary format ===
''Extra data'' is a binary blob, that contains additional information for the entries. For example, if an entry is an Asset Bundle entry, {{verify|What does an Asset Bundle entry mean exactly?}} the entry's <code>extra_data_offset</code> field points to a structure in this blob to determine the [[asset bundle name]], the [[asset bundle hash]], as well as download options for the asset bundle.


{{DataStructure|
{{DataStructure|
Line 110: Line 119:
       {{DataStructureField|class_name_length|uint8|1-byte integer, length of the following string.}}
       {{DataStructureField|class_name_length|uint8|1-byte integer, length of the following string.}}
       {{DataStructureField|class_name|string|UTF-8 string, the length is stored in the <code>class_name_length</code> field.}}
       {{DataStructureField|class_name|string|UTF-8 string, the length is stored in the <code>class_name_length</code> field.}}
       {{DataStructureField|payload|sized string struct|A little-endian UTF-16 string containing [[Wikipedia:JSON|JSON]] data. The JSON structure is described below. See [[#Sized string binary format]] and [[#AssetBundleRequestOptions object structure]] for details.}}
       {{DataStructureField|payload|sized string struct|A [[wikipedia:little endian|little endian]] UTF-16 string containing [[wikipedia:JSON|JSON]] data. The JSON structure is described below. See [[#Sized string binary format]] and [[#AssetBundleRequestOptions object structure]] for details.}}
     }}
     }}
   }}
   }}
Line 117: Line 126:
==== AssetBundleRequestOptions object structure ====
==== AssetBundleRequestOptions object structure ====


This is a [[Wikipedia:JSON|JSON]] document embedded within the extra data entry. It provides more information about downloadable asset bundles, for example, the [[asset bundle name]], [[asset bundle hash]], the size of the bundle and, other information.
''AssetBundleRequestOptions'' is a [[wikipedia:JSON|JSON]] document embedded within the extra data entry. It provides more information about downloadable asset bundles, for example, the [[asset bundle name]], [[asset bundle hash]], the size of the bundle and, other information.


{{DataStructure|
{{DataStructure|
Line 139: Line 148:
   {{DataStructureField|length|uint32|4-byte integer, determines the amount of bytes in the <code>text</code> field.}}
   {{DataStructureField|length|uint32|4-byte integer, determines the amount of bytes in the <code>text</code> field.}}
   {{DataStructureField|text|array|Byte array with the contents of the string. The exact format (UTF-8 vs UTF-16) must be determined separately.|
   {{DataStructureField|text|array|Byte array with the contents of the string. The exact format (UTF-8 vs UTF-16) must be determined separately.|
     {{DataStructureField||uint8|A byte within the string.}}
     {{DataStructureField||uint8|A byte of the string.}}
   }}
   }}
}}
}}