uecastoc

package module
v0.0.0-...-5d7c77b Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 21, 2023 License: GPL-3.0 Imports: 14 Imported by: 0

README

utoc File Format

This document shows the file format of utoc files, and they show their relation with the .ucas files for the Unreal Engine. This has been reverse engineered based on the game files of Grounded, and the source code of the Unreal Engine. That being said, let's dive right into it.

The UTOC struct is made up of the following properties:

FIoStoreTocHeader - Header
int64 - TocFileSize
TArray<FIoChunkId> ChunkIds
TMap<FIoChunkId, int32> ChunkIdToIndex
TArray<FIoStoreTocCompressedBlockEntry> CompressionBlocks
TArray<FIoStoreTocEntryMeta> ChunkMetas
TArray<FName> Compression

First the header: For any extra data structure used in the header, an indent is made. Everything in the file is Little Endian.

FILE HEADER (FIoStoreTocHeader), total bytes: 144
    uint8 {1}[16]           - static expression of 16 bits for MAGIC word (string: "-==--==--==--==-") 
    uint8 {1}           - Version - Current options are Initial(1), DirectoryIndex(2), PartitionSize(3)
    uint8 + uint16 {3}  - Reserved/Padding
    uint32 {4}          - Header Size (144)
    uint32 {4}          - Entry Count
    uint32 {4}          - Count of Compressed Blocks 
    uint32 {4}          - Entry size of Compressed Block (12)
    uint32 {4}          - Count Compression method names 
    uint32 {4}          - Compression method name length (32)
    uint32 {4}          - Compression block size (0x10000)
    uint32 {4}          - Directory index Size
    uint32 {4}          - Partition Count (0) // default value is 0
    uint64 {8}          - Container ID (FIoContainerID)
    FGUID {16}          - Guid Encryption Key
        uint32 {4}      - A
        uint32 {4}      - B
        uint32 {4}      - C
        uint32 {4}      - D
    uint8 {1}           - Container flags - uint8 bitmask enum - see https://docs.unrealengine.com/5.1/en-US/API/Runtime/Core/IO/EIoContainerFlags/
    uint8 + uint16      - Reserved/Padding caused by bitmask
    uint8 [60]          - static 60 bytes of padding - "partition size" is among them as well, but seems unused...

Following the header the total TOC file size is serialized:

    int64 {4}           - TOC File Size

Based on the header, there are several number of data structures parsed. The way in which these data structures are parsed and how often they occur are listed here, in the form: "(data structure): {name of variable}" The individual data structures can be found below.

(CHUNK_ID)            : {Entry Count}
(OFFSET_AND_LENGTHS)  : {Entry Count}
(COMPRESSION_BLOCK)   : {Count of Compressed Blocks}
(byte {32})           : {Count Compression method names} // the 32 is derived from the compression method name length
(DIRECTORY_INDEX)     : {1} // number of bytes in data structure: Directory index Size
(CHUNK_META)          : {Entry Count}

Note: while there are Count Compression method names entries in some kind of array with the compression methods, this array should start at 1, as index 0 means "No compression method".

After the chunk meta has been read, there should not be any bytes left to parse from the .utoc file.

Data structures:

CHUNK_ID, total bytes: 12 - Hash made up of 12 uint8 bytes
    uint64 {8}          - ID
    uint16 {2}          - Index // seems to be always 0
    uint8  {1}          - Padding // making it 0x10 aligned
    uint8  {1}          - Type - See: https://docs.unrealengine.com/4.26/en-US/API/Runtime/Core/IO/EIoChunkType/

This hash identifies one chunk. As far as I know, the ID is invalid if the value is 0. Otherwise it seems to be random. I am not sure what the Index means, as this value is always 0. Maybe this is relevant if the files are too large to be described by one chunk. In that case you can have multiple chunks describing a file, possibly with multiple indices, but this is all speculation. The Type field corresponds to the EIoChunkType of the unreal engine. The values I've seen are 2, 3 and 10.

Value 2 means "BulkData" and is used to represent .uasset files. Value 3 means "OptionalBulkData" and is used to represent .ubulk files. The .uasset files all have unique chunk IDs, whereas the .ubulk files have an ID that was already taken by a .uasset file. The very last chunkID always has value 10, which means "PackageStoreEntry", but I have no clue what this represents. The Unreal Engine code isn't too clear about it. I suspect that this has something to do with the FPackageStoreEntry struct.

To investigate what it could mean, I set everything of those sections to 0 and checked with the UnrealPak Viewer tool (https://github.com/jashking/UnrealPakViewer). In the one with everything set to zero, it has "dependency counts" at 0, and the class entries are set to "uexp" or "ubulk". For the other version, the one with data, it has various "dependency counts", and the class entries are set to different things, such as "Texture2D", "StaticMesh", "Skeleton", "SoundWave" etc. Furthermore, the "ExportObjects" field in the empty one has counts 0, while the one with data does not.

Update: I have found a few more things about this. If the UTOC header version is 2, then the first entry in the list of chunk IDs has type 10, which is not "mapped" to a file. If the UTOC header version is 3, then this is the last entry. The compression blocks that the corresponding offset and lengths entry corresponds to, are compressed (or not), and when decompressed, they form a file with some special file format. I have dubbed this file format as "depsFile", and its specification can be found below.

OFFSET_AND_LENGTHS, total bytes: 10
    byte {5}            - Offset
    byte {5}            - Length

Read as a uint64, with only the lower 5 bytes set, the others should be 0. There are just as many of these entries as Chunk IDs and file entries. The offset and lengths actually show the data of the individual files that are compressed in the .ucas file. Note that all data must be decompressed first.

NOTE: The length of the offlengths show the length of the decompressed file! The offset follows directly after the next length, but it's aligned to 0x10000. Therefore, there may be large gaps between them. This is done for indexing in the array of chunks.

COMPRESSION_BLOCK, total bytes: 12
    byte {5}            - Offset
    byte {3}            - Compressed block size
    byte {3}            - Uncompressed block size
    uint8 {1}           - Compression method

Read offset as uint64, with only lower 5 bytes set. The others should be read as uint32, with only the lower 3 bytes set. The compression method is interpreted as an index in the list of compression method names.

NAME_Zlib
NAME_Gzip
NAME_LZ4
"Bogus"
"Oodle"

The offset is an offset in the .ucas field, and it is always 16-aligned. The compression block size is the length of the data in the .ucas field. If the compression block size equals the uncompressed block size, it is obviously not compressed. If it is, the compression method is an index in the list of compression method entries. One note: in case of 0, this is always "None".

As the offset values are always 16-aligned, there are bogus values in between the parts that are unallocated. I don't know the significance of these values, but I think they are just random.

DIRECTORY_INDEX (total bytes is variable)
    uint32 {4}          - Mount point string length (includes nullbyte)
    byte {X}            - Mount point string
    uint32 {4}          - Directory Index array Entry Count (DIEC)
    DIR_INDEX_ENTRY {DIEC * 16} // an array of DIR_INDEX_ENTRY
        uint32 {4}      - Directory Name index
        uint32 {4}      - First child entry index
        uint32 {4}      - First sibling entry index
        uint32 {4}      - First file entry index
    uint32 {4}          - File Index array Entry Count (FIEC)
    FILE_INDEX_ENTRY {FIEC * 12} // an array of FILE_INDEX_ENTRY
        uint32 {4}      - File name index
        uint32 {4}      - Next file entry index
        uint32 {4}      - UserData (index, I guess)
    uint32 {4}          - number of strings of the string array 
    FSTRING {number_of_strings * (4 + X)} // an array of strings, referred to as StringTable
        uint32 {4}      - string length, nullbyte included
        byte {X}        - actual string

The directory index entries and the file index entries have a "name" index, which is used to refer to a name in the StringTable. The child and sibling indices for the Directory entries point to a different Directory entry in the same array. The file entry indices point to the file entry in the FILE_INDEX_ENTRY array.

CHUNK_META, total bytes: 33
    byte {32}        - Hash of chunk (This is standard SHA1 hash of 20 bytes, with 12 bytes of padding)
    uint8 {1}        - Flags

Possible flags are: NoneMetaFlag (0), CompressedMetaFlag (1), MemoryMappedMetaFlag (2) However, in the Grounded utoc file, the flag value is always 1.

depsFile Format

This still belongs to the .utoc file format, but it's encoded in the .ucas file. In the .ucas file, either the first large chunk of data (in header version 2), or the last chunk of data (in version 3) is specified as follows:

depsFile HEADER, total bytes: 32
    uint64 {8}      -   Chunk ID of this "file"
    uint64 {8}      -   Number of IDs (nID)
    uint32 {4}      -   sizeof(chunkID) (always 8)
    uint32 {4}      -   as of now, an unknown value
    byte   {4}      -   zero bytes PADDING
    uint32 {4}      -   Number of IDs (again, but now uint32?)

The header is followed by an ORDERED list of Chunk IDs, of size nID. The list is unique, and it contains all of the IDs that are present in the .utoc file. I suspect that this is done for performance, for quick searching. There's just one Chunk ID missing, and that's the thing with the chunk type being 10 (so the one that contains this "depsFile Format").

The list is immediately followed by a uint32 value describing the size in bytes of the rest of the file. It excludes the trailling (8) zerobytes.

The "rest of the file" consists of the following very interesting information. First, there are blocks of 32 bytes, with data that links .uasset files to other files. These are the connections that are made between dependencies, and they show how many exports each entry has. For every single ID that was provided in the previous part with the chunk IDs, there is a corresponding entry with this structure;

depsFile DEPENDENCIES, total bytes: 32
    uint64 {8}      - File size (uncompressed)
    uint32 {4}      - Number of Export Objects
    uint32 {4}      - A number either 1, 2 or 3. Not sure what this means

    uint64 {8}      - Index of some sort; unique and starting at 0, but skipping some numbers
    uint32 {4}      - Number of Dependency Packages
    uint32 {4}      - Offset to dependency packages

The list of these dependencies is followed by a long list of chunk IDs, which are in fact the dependencies. The offset in a dependency entry points to the start of such chunk ID, but these offsets are a bit tricky. This offset is taken from the specific offset entry, before reading the entry, so all offsets jump over the whole list. The number of dependency packages tells how far one must read! I think this could have been done a bit more efficiently, but unfortunately I didn't design this file format.

The number of IDs in the dependency area can be calculated. The length of the "rest of the file" is known, which consists of a number of dependency-entries and the IDs. We know how many dependency-entries there are, so we can calculate how many IDs there are. The list of IDs is followed by 8 nullbytes, which dictates the end of the file.

UAsset File Format

After unpacking the .ucas file using the .utoc file, I found out that the .uasset files that were created differ quite from the original .uasset files. They used to be different after unpacking with the .pak files. Therefore, I decided to figure out the (new) .uasset file format after unpacking.

A notable change is that when unpacking, the unreal signature (C1 83 2A 9E) is omitted. The version number is omitted as well. I think this is done to preserve space, as these are probably constant among all .uasset files in the .ucas file.

In addition to this, there is no split between .uasset and .uexp anymore; everything is concatenated within one .uasset file. This is also the intended way in which .uasset files are stored, I believe. It's just useful to split the two in some cases.

The uasset header is divided into two parts; a constant file header (of size 64), and a variable sized part. After the complete .uasset header, the contents of the would be .uexp file starts.

Note: most of this information is reverse-engineerd by looking at the information in the .ucas viewer linked before.

UASSET HEADER, total bytes: 64
    uint64 {8}      - Name in FMappedName format.  See https://docs.unrealengine.com/5.0/en-US/API/Runtime/Core/Serialization/FMappedName/
    uint64 {8}      - SourceName in FMappedName format
    uint32 {4}      - "Package Flags" Generally constant at (00 00 00 80).  See https://docs.unrealengine.com/5.1/en-US/API/Runtime/CoreUObject/UObject/EPackageFlags/ 
    uint32 {4}      - CookedHeaderSize - This is the total header size, if it were an old .uasset file, so it deviates
    int32 {4}      - Name Map Offset
    int32 {4}      - Name Map Size (in bytes)
    int32 {4}      - Name Map Hashes offset
    int32 {4}      - Name Map Hashes Size (in bytes)
    int32 {4}      - Import Map Offset
    int32 {4}      - Export Map offset
    int32 {4}      - Export Bundles Offset
    int32 {4}      - Graph Data Offset / Dependency Packages offset (also duplicated in dependency file in .ucas)
    int32 {4}      - Graph Data Size / Dependency Package Size (in bytes)
    int32 {4}      - Padding

The header indicates a lot of offsets of all the parts in the .uasset file. The remainder of this section highlights each of these parts in order. Sometimes, the header lists the length of a few parts, but I'm not sure why it does this, as all lengths can be inferred from the other offsets, except maybe the list of strings.

Directly after the file header, there is one nullbyte, after which the name directory index is shown.

NAMES DIRECTORY
for each name
    uint8 {1}       - Name length (EXCLUDING null terminator)
    byte  {X}       - Name
    byte  {1}       - null terminator

Due to the strings in the Names Directory being of different sizes, it is possible that the byte count is not aligned anymore. After the Names Directory, there are nullbytes to make it aligned to 8 bytes. This is probably done for performance reasons.

String Hashes

The names directory is followed by a list 64-bit hashes, representing the hashes of the strings. The very first value is the "AlgorithmID" of the hash. This value seems to be always 00 00 64 C1 00 00 00 00, and this corresponds to this line of code in the Unreal Engine source code. The hashing algorithm is the CityHash64 algorithm, and it works on strings that were made to be lowercase. So to get the value, make the string lowercase and hash it with the CityHash64 algorithm.

Import Objects

The list of string hashes is followed by a list of Import Object IDs. This is not the same as the file IDs, and I'm not sure what these values represent. Each value is 8 bytes, and if the import name is "None", the value is 0xFFFFFFFFFFFFFFFF, or -1. Not sure what to do with this, but the number of entries correspond to the number of import objects in the ucas file viewer.

In Helios' tool, the Import Objects correspond to the Linked Classes.

Export Objects

The next area of the file, are Export Objects. These are stored in a specific file structure, shown below.

EXPORT OBJECTS
    uint64 {8}                   - Cooked Serial Offset
    uint64 {8}                   - Cooked Serial Size
    FMappedName {4}              - Object Name
    FPackageObjectIndex {8}      - Outer Index
    FPackageObjectIndex {8}      - Class Index
    FPackageObjectIndex {8}      - Super Index
    FPackageObjectIndex {8}      - Template Index
    FPackageObjectIndex {8}      - Global Import Index
    uint64 {8}                   - Public Export Hash
    EObjectFlags {1}             - Object Flags
    EExportFilterFlags {1}       - Client/Server Flags
    byte {2}                     - Padding

The fields that were used are mainly copied from the .ucas file viewer. In any case, the offsets that are described is an integer describing the offset in the name directory, or the list of strings.

The part after the .uasset file header is built up from different ExportObjects. When there are multiple ExportObjects, they are consecutive, which the SerialOffset and sizes illustrate. For the very first ExportObject, the Serial Offset field is the exact same value as the "total header size" in the header of the file. Now in this new file format, this doesn't really hold anymore, as it differs from the regular file. However, the serial offset states where the .uexp portion of the file starts. The Serial Size also has great significance, as this is the length of the .uexp portion of the file.

The object name offset and the class name offset probably have some use as well, but these probably refer to the old file format. Therefore, the offset is off and I'm not sure how to correct it right now. Changing files is probably mostly impacting the serial size and the serial offset, so I'll keep this as a mystery for now.

ExportObject metadata?

The export objects are followed by a strange data structure. It first starts with a 64-bit number, which seems to have no relation to the rest. Afterwards, there is an enumeration of some kind of metadata to the export objects. The metadata consists of four 32-bit numbers, and there are exactly as many metadata entries as export objects. This is still a mystery, but I don't think I'll touch ExportObjects anyway.

Dependency packages

After all export objects information, there is a list of dependency packages. It starts with a uint32 stating how many dependency entries there are. After this, the following dependencies are shown;

DEPENDENCY PACKAGE
    uint64 {8}      - ID
    uint32 {4}      - Number of Entries (?) // Not quite sure, it's often just 1, sometimes 2 and 3.
    int32 {4}       - is present*
    uint32 {4}      - mostly one // not sure what it is, but almost always 1.
    
    if (number of entries > 1){
        uint32 {4 * (2 * (numberOfEntries - 1))}**
    }

* If the value of "is present" is 0, then the ID listed is included in the current .ucas file. If this value is -1, it's not included in the current .ucas file.

** If the number of entries is 2 or 3, there are more numbers defined. For every number larger than 1, there are two additional uint32 numbers in a dependency entry.

It's quite clear that I am still not sure what all of the fields mean and how they relate to each other. However, after the list of the dependencies, the entire former .uexp file starts!

End of the header

This concludes the .uasset header part of the file. There are quite some values that are still unknown to me. However, most of these values will probably not change based on the created mods. I can only hope that I know enough to do the most interesting stuff with!

Documentation

Index

Constants

View Source 
const (
	EIoChunkTypeBulkData         = 2
	EIoChunkTypeOptionalBulkData = 3
)
View Source 
const (
	// these are the types of Export Objects, with special thanks to Helios
	EOTypeFunction       EOType = 1
	EOTypeEnum                  = 3
	EOTypeGeneratedClass        = 9
	EOTypeTable                 = 11
	EOTypeMeshComponent         = 41
	EOTypeObject                = 49
)
View Source 
const (
	MagicUtoc       string = "-==--==--==--==-"
	VersionUtoc     uint32 = 3
	LegacyUtoc      uint32 = 2
	UnrealSignature string = "\xC1\x83\x2A\x9E"
	MountPoint      string = "../../../"
	NoneEntry       uint32 = 0xffffffff
	PackFileName    string = "Packed_P"
)
View Source 
const (
	NoneContainerFlag       EIoContainerFlags = iota
	CompressedContainerFlag                   = iota
	EncryptedContainerFlag                    = (1 << iota)
	SignedContainerFlag                       = (1 << iota)
	IndexedContainerFlag                      = (1 << iota)
)
View Source 
const (
	CompSize = 0x10000
)

Variables

View Source 
var (
	ErrWrongMagic         = errors.New("error: magic number was not found")
	ErrUnknownUtocVersion = errors.New("error: utoc version is unknown")
	ErrContainerFlag      = errors.New("WARNING: the container flags were not 0 nor the indexed flag, so it could be a special case")
	ErrExpectedBytesRead  = errors.New("expected to have read all bytes")
	ErrTocEntryBounds     = errors.New("TocEntry out of container bounds")
)
View Source 
var ParsedDataPath string = "./Runtime/"

Functions

func PackDirectory 

func PackDirectory(dirPath string)

PackDirectory builds the directory structure recursively of the entire directory Then, it packs the retrieved structure into a UCAS/UTOC file combination.

Types

type ArrayProperty 

type ArrayProperty struct {
	ArrayType       string `json:"arraytype"`
	ArrayStructType string `json:"arraystructtype,omitempty"` // only if ArrayType == "StructProperty"
	Size            int    `json:"arraysize"`
	Children        []any
}

func (ArrayProperty) GetName 

func (x ArrayProperty) GetName() string

type BaseProperty 

type BaseProperty struct {
	Value     string     `json:"baseproperty"`
	Secondary int32      `json:"secondary,omitempty"`
	Children  []Property `json:"properties"`
}

BaseProperty is the same as the regular Property except, its children are Properties

type BoolProperty 

type BoolProperty struct {
	Value bool `json:"bool"`
}

func (BoolProperty) GetName 

func (x BoolProperty) GetName() string

type ByteProperty 

type ByteProperty struct {
	Value byte `json:"byte"`
}

type CasToc 

type CasToc struct {
	Root        *Directory
	ContainerID uint64
}

func ParseUTocFile 

func ParseUTocFile(filepath string) (ct CasToc, err error)

func (*CasToc) PathToDirectory 

func (ct *CasToc) PathToDirectory(path string) (*Directory, error)

traverses the path starting from the mountpoint (../../../) and returns the directory pointer 

type DepLinks struct {
	FileSize      uint64 // uncompressed file size
	ExportObjects uint32 // number of Export Objects
	MostlyOne     uint32 // this is either 1, 2 or 3, but most often 1.

	// All entries have a unique value for this field, starting at 0
	// It does skip some numbers, so I'm not sure what this means.
	// Looks like some kind of index though
	SomeIndex          uint64
	DependencyPackages uint32 // number of dependency packages that this connection has
	Offset             uint32 // an offset to point to the dependency packages
}

type Dependencies 

type Dependencies struct {
	ThisPackageID         uint64 `json:"packageID"`
	ChunkIDToDependencies map[uint64]FileDependency
}

Dependencies contains data extracted from the dependencies section in the .ucas file. An instance of this will be used to convert back to this section.

func ParseDependencies 

func ParseDependencies(b []byte) *Dependencies

func (*Dependencies) Deparse 

func (d *Dependencies) Deparse() *[]byte

Deparses the Dependencies struct exactly as how it was parsed This was checked using a simple diff tool.

type DepsHeader 

type DepsHeader struct {
	ThisPackageID    uint64
	NumberOfIDs      uint64
	IDSize           uint32  // always 8
	Padding          [4]byte // value is in hex, LE, 00 00 64 C1
	ZeroBytes        [4]byte
	NumberOfIDsAgain uint32 // whether this means an offset or?
}

type Directory 

type Directory struct {
	Name      string
	ChildDirs []*Directory
	Files     []*FileInfo
	Parent    *Directory
}

func (*Directory) UnpackDirectory 

func (d *Directory) UnpackDirectory(openUCASfile *os.File, unpackPath string)

unpacks the directory inside the unpackPath; does NOT include the entire path to the root

type EIoContainerFlags 

type EIoContainerFlags uint8

type EOType 

type EOType uint16

type EnumProperty 

type EnumProperty struct {
	BaseEnum string `json:"enumBase"`
	Value    string `json:"enumValue"`
}

the rest are "terminal" structures

func (EnumProperty) GetName 

func (x EnumProperty) GetName() string

type ExportObject 

type ExportObject struct {
	SerialOffset     uint64
	SerialSize       uint64
	ObjectNameOffset uint32 // index in the Names Directory list; name of this "file"
	SomeNumber       uint32
	ClassNameOffset  uint64
	FileHashes       [4]uint64
	Type             uint16
	OtherValues      [3]uint16
}

total bytes: 72

type FGuid 

type FGuid struct {
	A, B, C, D uint32
}

four components that make up a unique identifier. Seems a bit overkill, but: https://github.com/EpicGames/UnrealEngine/blob/release/Engine/Source/Runtime/Core/Public/Misc/Guid.h

type FIoChunkHash 

type FIoChunkHash struct {
	Hash    [20]uint8 //SHA1
	Padding [12]uint8
}

type FIoChunkID 

type FIoChunkID struct {
	ID      uint64
	Index   uint16
	Padding uint8
	Type    uint8
}

type FIoContainerID 

type FIoContainerID uint64

type FIoDirectoryIndexEntry 

type FIoDirectoryIndexEntry struct {
	Name             uint32
	FirstChildEntry  uint32
	NextSiblingEntry uint32
	FirstFileEntry   uint32
}

type FIoDirectoryIndexResource 

type FIoDirectoryIndexResource struct {
	MountPoint       FString
	DirectoryEntries []FIoDirectoryIndexEntry
	FileEntries      []FIoFileIndexEntry
	StringTable      []FString
}

type FIoFileIndexEntry 

type FIoFileIndexEntry struct {
	Name          uint32
	NextFileEntry uint32
	UserData      uint32
}

type FIoOffsetAndLength 

type FIoOffsetAndLength struct {
	Offset [5]uint8
	Length [5]uint8
}

func (*FIoOffsetAndLength) GetLength 

func (f *FIoOffsetAndLength) GetLength() uint64

func (*FIoOffsetAndLength) GetOffset 

func (f *FIoOffsetAndLength) GetOffset() uint64

func (*FIoOffsetAndLength) SetLength 

func (f *FIoOffsetAndLength) SetLength(length uint64)

func (*FIoOffsetAndLength) SetOffset 

func (f *FIoOffsetAndLength) SetOffset(offset uint64)

type FIoStoreTocCompressedBlockEntry 

type FIoStoreTocCompressedBlockEntry struct {
	Offset            [5]uint8
	CompressedSize    [3]uint8
	UncompressedSize  [3]uint8
	CompressionMethod uint8
}

func (*FIoStoreTocCompressedBlockEntry) GetCompressedSize 

func (f *FIoStoreTocCompressedBlockEntry) GetCompressedSize() uint32

func (*FIoStoreTocCompressedBlockEntry) GetOffset 

func (f *FIoStoreTocCompressedBlockEntry) GetOffset() uint64

func (*FIoStoreTocCompressedBlockEntry) GetUncompressedSize 

func (f *FIoStoreTocCompressedBlockEntry) GetUncompressedSize() uint32

func (*FIoStoreTocCompressedBlockEntry) SetCompressedSize 

func (f *FIoStoreTocCompressedBlockEntry) SetCompressedSize(size uint32)

func (*FIoStoreTocCompressedBlockEntry) SetOffset 

func (f *FIoStoreTocCompressedBlockEntry) SetOffset(offset uint64)

func (*FIoStoreTocCompressedBlockEntry) SetUncompressedSize 

func (f *FIoStoreTocCompressedBlockEntry) SetUncompressedSize(size uint32)

type FIoStoreTocEntryMeta 

type FIoStoreTocEntryMeta struct {
	ChunkHash FIoChunkHash
	Flags     FIoStoreTocEntryMetaFlags
}

type FIoStoreTocEntryMetaFlags 

type FIoStoreTocEntryMetaFlags uint8
const (
	NoneMetaFlag FIoStoreTocEntryMetaFlags = iota
	CompressedMetaFlag
	MemoryMappedMetaFlag
)

type FIoStoreTocResourceInfo 

type FIoStoreTocResourceInfo struct {
	Header               UTocHeader
	TocFileSize          uint64
	ChunkIDs             []FIoChunkID
	IDToChunk            map[uint64]FIoChunkID
	ChunkIDToIndex       map[FIoChunkID]uint32
	ChunkIDToOfflengths  map[FIoChunkID]FIoOffsetAndLength
	CompressionBlocks    []FIoStoreTocCompressedBlockEntry
	ChunkMetas           []FIoStoreTocEntryMeta
	ChunkBlockSignatures []FSHAHash
	CompressionMethods   []FName
	DirectoryResource    FIoDirectoryIndexResource
	// DirectoryIndexBuffer []byte
	Offlengths []FIoOffsetAndLength
}

type FName 

type FName string

type FSHAHash 

type FSHAHash struct {
	Hash [20]byte
}

func (*FSHAHash) ToString 

func (f *FSHAHash) ToString() string

hash would be generated by FSHA1

type FString 

type FString string // just a string I guess? But fancier...

type FileDependency 

type FileDependency struct {
	FileSize      uint64   `json:"uncompressedSize"`
	ExportObjects uint32   `json:"exportObjects"`
	MostlyOne     uint32   `json:"requiredValueSomehow"`
	SomeIndex     uint64   `json:"uniqueIndex"`
	Dependencies  []uint64 `json:"dependencies"` // lists the ID of each dependency
}

type FileInfo 

type FileInfo struct {
	Name          string
	ID            uint64
	Parent        *Directory
	Dependencies  []uint64
	ExportObjects uint32

	FileHash FIoChunkHash
	// contains filtered or unexported fields
}

type FloatProperty 

type FloatProperty struct {
	Value float32 `json:"float"`
}

func (FloatProperty) GetName 

func (x FloatProperty) GetName() string

type Int64Property 

type Int64Property struct {
	Value int64 `json:"int64"`
}

func (Int64Property) GetName 

func (x Int64Property) GetName() string

type IntProperty 

type IntProperty struct {
	Value int32 `json:"int"`
}

func (IntProperty) GetName 

func (x IntProperty) GetName() string

type NameInterface 

type NameInterface interface{ GetName() string }

These interfaces are for finding the "None" value while parsing

type NameProperty 

type NameProperty struct {
	Value          string `json:"name"`
	SecondaryValue int32  `json:"secondary,omitempty"`
}

func (NameProperty) GetName 

func (x NameProperty) GetName() string

type ObjectProperty 

type ObjectProperty struct {
	Value int32 `json:"object"`
}

func (ObjectProperty) GetName 

func (x ObjectProperty) GetName() string

type Property 

type Property struct {
	Value     string `json:"property"`
	Secondary int32  `json:"secondary,omitempty"`
	Child     any    `json:"child,omitempty"`
}

each regular property has a child that is another structure property or a "terminal"

func (Property) GetName 

func (x Property) GetName() string

type SoftObjectProperty 

type SoftObjectProperty struct {
	Value     string `json:"softproperty"`
	Secondary int32  `json:"secondary,omitempty"`
}

type StructProperty 

type StructProperty struct {
	StructType string `json:"structtype"`
	Children   []any  `json:"children"`
}

keep parsing until "None" all properties that come in between are children of this Property

func (StructProperty) GetName 

func (x StructProperty) GetName() string

type UAssetHeader 

type UAssetHeader struct {
	RepeatNumber             [2]uint64 // same number repeated twice
	PackageFlags             uint32    // value will be 0x80000000
	TotalHeaderSize          uint32    // of the original file header size; but that included the dependencies...
	NamesDirectoryOffset     uint32    // points to nullbyte, so do +1
	NamesDirectoryLength     uint32    // length is in bytes
	NamesHashesOffset        uint32    // first entry is always the algorithm ID
	NamesHashesLength        uint32    // length is in bytes
	ImportObjectsOffset      uint32
	ExportObjectsOffset      uint32 // offset to some extra header information
	ExportMetaOffset         uint32 // points to some memory, just store the either 24 or 32 bytes.
	DependencyPackagesOffset uint32 // first value is a uint32, number of dependency packages.
	DependencyPackagesSize   uint64
}

static header with the first 64 bytes of the .uasset file

type UAssetResource 

type UAssetResource struct {
	Header        UAssetHeader
	NamesDir      []string
	ExportObjects []ExportObject
	// the rest of the file probably won't change in mods
	ImportObjects      []byte
	ExportMeta         []byte
	DependencyPackages []byte

	ParsedJSON *[]byte
}

func ParseUAssetFile 

func ParseUAssetFile(path string) (uasset UAssetResource, err error)

ParseUAssetFile takes a string and it expects the full .uasset file, including .uexp If the .uexp isn't there, there might be problems. Not all data must be parsed; only the parts that may be changed due to modding. That's what I'm doing here.

func (*UAssetResource) DeparseUAssetFile 

func (uasset *UAssetResource) DeparseUAssetFile(json *[]byte) *[]byte

func (*UAssetResource) PrintNamesDirectory 

func (u *UAssetResource) PrintNamesDirectory()

type UTocData 

type UTocData struct {
	IDs        []FIoChunkID
	Offlengths []FIoOffsetAndLength
	CBlocks    []FIoStoreTocCompressedBlockEntry
	Metas      []FIoStoreTocEntryMeta
	BlockSize  uint32
	Deps       *Dependencies
	FNameToID  map[string]uint64
}

type UTocHeader 

type UTocHeader struct {
	Magic                       [16]byte
	Version                     uint32 // they state it's uint8, but I doubt it
	HeaderSize                  uint32
	EntryCount                  uint32
	CompressedBlockEntryCount   uint32
	CompressedBlockEntrySize    uint32 // they say "For sanity checking"
	CompressionMethodNameCount  uint32
	CompressionMethodNameLength uint32
	CompressionBlockSize        uint32
	DirectoryIndexSize          uint32
	PartitionCount              uint32 // should be 0
	ContainerID                 FIoContainerID
	EncryptionKeyGuid           FGuid
	ContainerFlags              EIoContainerFlags
	Padding                     [63]byte
}

useful: https://github.com/jashking/UnrealPakViewer/blob/master/PakAnalyzer/Private/IoStoreDefines.h Unreal Docs: https://github.com/EpicGames/UnrealEngine/blob/99b6e203a15d04fc7bbbf554c421a985c1ccb8f1/Engine/Source/Runtime/Core/Private/IO/IoStore.h

type UexpDataTable 

type UexpDataTable struct {
	Name           string         `json:"name"`
	Header         []Property     `json:"datatableheader"`
	BaseProperties []BaseProperty `json:"datatable"`
}

implements UexpStructure

func (UexpDataTable) GetType 

func (u UexpDataTable) GetType() EOType

type UexpStructure 

type UexpStructure interface {
	GetType() EOType
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.