Recording file format

Motion Capture saves recordings in custom binary format. These files can be found in (world directory)/mocap_files/recordings and have .mcmocap_rec extension.

File structure can be divided into 3 parts (in order):

• version number
• header
• actions

General concepts

Motion Capture file format is designed to be compact, yet not compressed and therefore simple to read by mod or any other implementation. Single recording file is designed to hold single recording, which concept was described in the introduction.

Recording file format aims to be backwards compatible, but not forward compatible. Note that it's backwards compatible with recordings from previous Motion Capture versions, but Motion Capture current implementation of this format may have issues with compatibility between Minecraft versions - especially with recordings recorded before Motion Capture 1.4, which instead of string block and item IDs were using numeric IDs. In latest version issues may still appear because of changing NBT data, data components or string IDs.

Backwards compatibility doesn't include experimental format versions, found e.g. in alpha/beta versions.

Basic data types

Numeric values are saved in big-endian order.

Basic data types used in recording file format (names aren't used in file format itself, but are used in this documentation and in Motion Capture code):

• byte - 1-byte signed integer
• short - 2-byte signed integer
• int - 4-byte signed integer
• float - 4-byte floating point number (IEEE-754 / binary32)
• double - 8-byte floating point number (IEEE-754 / binary64)
• boolean - value 0 (false) or 1 (true) stored on one byte; other values should not be used

When referring to array of values with specified type square brackets will be used with size of an array in them, e.g. double[3] for array of 3 doubles or int[size] for array of ints with size equal to value of field size.

More complex but also commonly used data types:

• packed_int - stores 4-byte signed integer on either 1 or 5 bytes. If value is between 0 and 254 inclusively, it will be stored as one unsigned byte, if value is equal or greater than 255, or negative, first byte will be set to 255 and value will be saved as signed integer. Values between 0 and 254 should not be stored in 5-byte form.

Examples:

Bytes	Value
00	0 (0x00)
A5	165 (0xA5)
FF FF 00 00 00	255 (0xFF)
FF 00 01 00 00	256 (0x100)
FF FF FF FF FF	-1 (0xFFFFFFFF)
FF 18 00 00 00	invalid

• string - sequence of characters encoded in UTF-8 with specified size (without null terminator)

String structure

Name	Type
length	int/packed_int
bytes	byte[length]

length type depends on header's flags1 value. If PACKED_SIZE_STRINGS is defined, packed_int will be used instead of full size int. Initial value shouldn't be a problem, as string doesn't appear before flags1.

• uuid - 128-bit number stored like other in big-endian order

Version number and file format history

Version number is saved on a one byte. If version is later than current one, recording isn't loaded by Motion Capture. If it is experimental it will be loaded as long as it doesn't correspond to later stable version than current one, although note that loading experimental may result in an error (this is especially the case with recordings recorded between 1.4-alpha-1 and 1.4-alpha-8). Generally version number doesn't affect much how recording is loaded with one small exception described in section about header.

Possible values and their meaning:

• 0 - undefined version number (as in not defined in file, but it's valid and should be read)
• 1 - 127 - stable version number
• -1 - -127 - experimental version number of a corresponding stable version
• -128 - not defined (as in not valid for now but might have meaning in the future)

Version history:

• 1 - Motion Capture 1.0 - 1.1.1
• 2 - Motion Capture 1.2 - 1.2.1
• 3 - Motion Capture 1.3 - 1.3.9 for Minecraft 1.20.4 and older
• 4 - Motion Capture 1.3.5 - 1.3.9 for Minecraft 1.20.6 and later
• 5 - Motion Capture 1.4 and later

Header

On versions 1 and 2 header consists only of startPos and startRot - flags1 is not present and should be assumed to be 0.

Names used in this and other tables usually refer to name in code (but they don't have to) and aren't part of file format - they're just here to make referring to them easier.

Name	Type	Condition
startPos	double[3]	always present
startRot	float[2]	always present
flags1	byte	version != 1 && version != 2
itemIdMap	ItemIdMap	(flags1 & HAS_ID_MAPS) != 0
blockStateIdMap	BlockStateIdMap	(flags1 & HAS_ID_MAPS) != 0
dimensionId	string	(flags1 & DIMENSION_SPECIFIED) != 0
assignedProfile	AssignedProfile	(flags1 & PROFILE_ASSIGNED) != 0
extensionHeaders	ExtensionHeaders	(flags1 & HAS_EXTENSIONS) != 0
experimentalSubversion	ExtensionHeaders	(flags1 & EXPERIMENTAL_SUBVERSION) != 0

Field descriptions:

[startPos] - initial position in world of a recorded player (x, y, z)

[startRot] - initial rotation of a recorded player (rotY, rotX)

[flags1] - flags packed into one byte. These flags are

Bit	Name	Mask
0	ENDS_WITH_DEATH*	0b00000001 (0x01)
1	PACKED_SIZE_STRINGS	0b00000010 (0x02)
2	HAS_ID_MAPS	0b00000100 (0x04)
3	DIMENSION_SPECIFIED	0b00001000 (0x08)
4	PROFILE_ASSIGNED	0b00010000 (0x10)
5	HAS_EXTENSIONS	0b00100000 (0x20)
5	EXPERIMENTAL_SUBVERSION	0b01000000 (0x40)

* - deprecated

ENDS_WITH_DEATH - when true, player dies and then is removed, instead of just being removed at the end of recording playback. This has been deprecated in version 1.4 (format version 5) and DIE action is used instead.

PACKED_SIZE_STRINGS - when true, recording uses packed strings instead of legacy strings (both string formats were described in basic data types section). Since version 1.4 value of this flag is always set to true.

HAS_ID_MAPS - specifies whenever recording uses and therefore contains ID maps. Since version 1.4 this is set to true when at least one of ID map contains at least one element (there is action that requires block states or item ID), otherwise it's false.

DIMENSION_SPECIFIED - specifies whenever recording has assigned dimension to it - see assign_dimension setting.

PROFILE_ASSIGNED - specifies whenever recording has player profile assigned to it - see assign_profile setting.

HAS_EXTENSIONS - specifies whenever recording contains extensions.

EXPERIMENTAL_SUBVERSION - specifies revision of experimental format.

[itemIdMap] - array of item string IDs with position in this array being their ID starting with 1, as ID 0 corresponds to air.

ItemIdMap structure

Name	Type
size	int
ids	string[size]

ids are in standard Minecraft format, that is namespace:id or optionally id when namespace is "minecraft".

[blockStateIdMap] - array of block states with ID numeric values working the same way as in itemIdMap.

BlockStateIdMap structure

Name	Type
size	int
blockState	BlockState[size]

BlockState structure

Name	Type
id	string
propertyCount	short
properties	BlockStateProperty[propertyCount]

id is block ID in standard Minecraft format.

BlockStateProperty

Name	Type
propertyName	string
propertyValue	string

propertyName can be for example "facing" and propertyValue "north". Property values are always converted and saved as string regardless of their type.

[dimensionId] - ID in standard Minecraft format of dimension in which recording was recorded.

[assignedProfile] - player profile assigned to the recording.

AssignedProfile

Name	Type	Condition
profileFlags	byte	always present
name	string	(profileFlags & HAS_NAME) != 0
id	uuid	(profileFlags & HAS_ID) != 0
skinValue	string	(profileFlags & HAS_SKIN_VALUE) != 0
skinSignature	string	(profileFlags & HAS_SKIN_SIGNATURE) != 0

profileFlags specify which profile fields are present:

Bit	Name	Mask
0	HAS_NAME	0b00000001 (0x01)
1	HAS_ID	0b00000010 (0x02)
2	HAS_SKIN_VALUE	0b00000100 (0x04)
3	HAS_SKIN_SIGNATURE	0b00001000 (0x08)

If profileFlags is set to 0, profile doesn't have any fields and should behave as if it wasn't assigned.

[extensionHeaders] - array of used extensions and their headers. Extensions are a way of adding custom actions by other mods using Motion Capture API. To properly read and play recording all of used extensions need to be present. For more details about extensions read their documentation. (TODO: add link)

ExtensionHeaders structure

Name	Type
size	byte
extension	ExtensionData

ExtensionData structure

Name	Type
id	string
version	short
header	custom header

id refers to simple string ID (without any namespaces), is set by mod using API and is used to identify extensions.

version of a specific extension is also set by mod using API, similar as with Motion Capture version, if version in file is higher than assigned to currently installed extension with the same ID, then recording should not be loaded, if version is the same, everything is fine and if lower, then backwards compatibility should be assumed. (TODO: what about version=0?)

header is custom header defined and handled by mod using Motion Capture API. It can and usually will be just empty.

Actions

(TODO: finish) (TODO: include custom actions)

ID	Name	State action
0	NEXT_TICK	no
1	MOVEMENT_LEGACY*	n/a**
2	HEAD_ROTATION*	n/a**
3	CHANGE_POSE	yes
4	CHANGE_ITEM	yes
5	SET_ENTITY_FLAGS	yes
6	SET_LIVING_ENTITY_FLAGS	yes
7	SET_MAIN_HAND	yes
8	SWING	yes***
9	BREAK_BLOCK	no
10	PLACE_BLOCK	no
11	RIGHT_CLICK_BLOCK	no
12	SET_EFFECT_COLOR*	n/a**
13	SET_ARROW_COUNT	yes
14	SLEEP	yes
15	PLACE_BLOCK_SILENTLY	no
16	ENTITY_UPDATE	no
17	ENTITY_ACTION	no****
18	HURT	no
19	VEHICLE_DATA*	n/a**
20	BREAK_BLOCK_PROGRESS	no
21	MOVEMENT	no*****
22	SKIP_TICKS	no
23	DIE	no
24	RESPAWN	no
25	CHAT_MESSAGE	no
26	SET_SPECTATOR	yes
27	DUMMY	no
28	CLOSE_CONTAINER	no
29	SET_EFFECT_PARTICLES	yes
30	SET_NON_PLAYER_ENTITY_DATA	yes
255	custom action	yes

* - deprecated

** - state actions before deprecation, but now normal actions

*** - although SWING is technically state action, it differs from other state actions - it isn't placed on recording beginning and contains additional checks on how often state is changing

**** - ENTITY_ACTION isn't state action by itself, but is used to store other actions (mainly state actions)

***** - MOVEMENT isn't technically a state action but works in a similar way - it is created each tick, compared with previous state and saved when differs, and its change triggers start of recording

[NEXT_TICK] - ends execution of actions in current tick. It doesn't contain any additional data.

[MOVEMENT_LEGACY] - changes entity position and rotation - deprecated (since format version 5).

Name	Type
position	double[3]
rotation	float[2]
isOnGround	boolean

position - position of an entity relative to last position (x, y, z)

rotation - absolute rotation of an entity in order rotX, rotY (reversed order compared to rotation in header)

isOnGround - is entity on ground

[HEAD_ROTATION] - changes entity head rotation - deprecated (since format version 5).

Name	Type
headRotY	float

headRotY - rotation of an entity head

[CHANGE_POSE] - changes pose of an entity

Name	Type
poseId	int

pose - ID of pose

ID	Name
0	unknown
1	STANDING
2	FALL_FLYING
3	SLEEPING
4	SWIMMING
5	SPIN_ATTACK
6	CROUCHING
7	DYING
8	LONG_JUMPING
9	CROAKING
10	USING_TONGUE
11	SITTING
12	ROARING
13	SNIFFING
14	EMERGING
15	DIGGING
16	SLIDING
17	SHOOTING
18	INHALING

ID 0 (unknown) although it should not appear, it's valid and should be treated as STANDING.

[CHANGE_ITEM] - changes visible entity items (in hands or armor).

Name	Type
firstByte*	byte
items	ItemData[...]

* - if first byte is non-negative, then it should be treated as part of items

ItemData structure

Name	Type
type	byte
id*	int
data*	string

* - may not be present

type possible values

ID	Name	Contains id	Contains data
0	NO_ITEM	no	no
1	ID_ONLY	yes	no
2	ID_AND_NBT	yes	yes
3	ID_AND_COMPONENTS	yes	yes

(TODO: describe how item data is exactly encoded)

Originally structure of this operation consisted of 6 ItemData elements but format version 4 added option to specify its size on a first byte. If first byte is non-negative then entire action structure (including first byte) should be read as array of 6 ItemData structures. If first byte is negative, then items size equals -firstByte, or 0 when firstByte equals -128.

Slot to which item should be assigned is based on a position within items array.

Pos.	Slot
0	MAINHAND
1	OFFHAND
2	FEET
3	LEGS
4	CHEST
5	HEAD
6	BODY

If array is longer, additional elements should be ignored. If array is shorter, missing slots should be treated as empty.

[SET_ENTITY_FLAGS] - changes entity shared flags - https://minecraft.wiki/w/Java_Edition_protocol/Entity_metadata#Entity (first byte in this table).

Name	Type
flags	byte

[SET_LIVING_ENTITY_FLAGS] - changes living entity shared flags - https://minecraft.wiki/w/Java_Edition_protocol/Entity_metadata#Living_Entity (first byte in this table).

Name	Type
flags	byte

[SET_MAIN_HAND] - changes entity main hand.

Name	Type
isRight	boolean

If isRight is true then right hand is set as main, otherwise left hand.

[SWING] - entity swings his hand.

Name	Type
isOffHand	boolean

If isOffHand is true, entity swings its offhand, otherwise it swings mainhand.

[BREAK_BLOCK] - breaks block.

Name	Type
previousBlockState	int
blockPos	int[3]

previousBlockState - ID of a block state before breaking in order to optionally initialize it on playback start. If recording doesn't use ID maps (check out HAS_ID_MAPS) it directly maps to numeric ID of a block state in Minecraft, which may change with different game version or even set of mods. If recording uses ID maps, it is mapped to blockStateIdMap entry. Regardless ID maps being used or not, 0 is mapped to air (although in case of breaking block it is valid but doesn't make much sense).

blockPos - position of a broken block (x, y, z)

[PLACE_BLOCK] - places block.

Name	Type
previousBlockState	int
newBlockState	int
blockPos	int[3]

previousBlockState - ID of a block state before that was replaced

newBlockState - ID of a block state of placed block

blockPos - position of a placed block (x, y, z)

[RIGHT_CLICK_BLOCK] - player right clicks block.

Most of these fields (except offHand) correspond to fields of Minecraft BlockHitResult structure.

Name	Type
pos	double[3]
blockPos	int[3]
direction	byte
inside	boolean
offHand	boolean

pos - position of hit

blockPos - position of block that was hit

direction - direction of hit

ID	Name
0	DOWN
1	UP
2	NORTH
3	SOUTH
4	WEST
5	EAST

inside - is player head inside a block

offHand - was clicked by offhand

[SET_EFFECT_COLOR] - sets entity particle color (minecraft:entity_effect) and abience - deprecated (since format version 5). SET_EFFECT_PARTICLES is now used instead.

Name	Type
color	int
ambience	boolean

color - if 0, entity doesn't have effect particles, if non-0, color of minecraft:entity_effect

ambience - sets effect ambience - reduces count of particles (is true when effect is applied by beacon)

[SET_ARROW_COUNT] - sets entity arrow and bee stinger count.

Name	Type
arrowCount	int
beeStingerCount	int

[SLEEP] - specifies if entity is sleeping or not and carries bed position if needed.

Name	Type
isSleeping	boolean
bedPosition*	int[3]

* - may not be present

isSleeping - true when entity is sleeping, false otherwise

bedPosition - position of bed on which entity is sleeping (x, y, z), present only if isSleeping is true

[PLACE_BLOCK_SILENTLY] - places block silently (without playing sound effect).

Its data structure is identical to PLACE_BLOCK.

[ENTITY_UPDATE] - is used to manage additional playback entities.

Name	Type
updateType	byte
entityId	int
nbt*	string
position*	double[3]

* - may not be present

updateType - type of update

ID	Name
0	NONE
1	ADD
2	REMOVE
3	KILL
4	HURT
5	PLAYER_MOUNT
6	PLAYER_DISMOUNT

ID 0 (NONE) although it should not appear, it's valid and should such update should be ignored.

Difference between REMOVE and KILL is that when removed entity should just disappear and when killed it should first show dying animation, and then it should be removed.

entityId - ID of entity to add or execute operation on. This ID is numeric value, it doesn't refer to entity type but instance of an entity in a recording (except main entity, which doesn't have ID assigned). NONE and PLAYER_DISMOUNT updates ignore entityId value, but it should be present.

nbt - SNBT containing full entity data with its type ID, present only for ADD update type

position - initial position of entity, present only for ADD update type

[ENTITY_ACTION] - wraps action to be executed on a specified additional entity.

Name	Type
entityId	int
action	Action

entityId - ID of entity to add or execute action on

action - action to be executed

Structure of wrapped action is the same as structure of any other action in the recording - consists of byte (opcode) and data. Action can also be custom action (added by other mods). Note that if entity wasn't created (e.g. it was filtered by entity filter), action should not be executed. Action should only contain actions strictly related to entities (usually state actions). It should not call actions controlling ticks (NEXT_TICK or SKIP_TICKS) - this may lead to undefined behavior. It should also not call DIE or RESPAWN, and should use instead ENTITY_UPDATE action.

[HURT] - hurts main entity.

Name	Type
unused	byte

unused - unused value reserved for future uses (currently always 0), should be ignored

Note that it is only used to hurt main playback entity (usually player). For hurting other playback entities ENTITY_UPDATE is used.

[VEHICLE_DATA] - changes some data for some entities (mainly for rideable) - deprecated (since format version 5). SET_NON_PLAYER_ENTITY_DATA is now used instead which has identical but more compact format.

Name	Type
used	boolean
byte1*	byte
flag1*	boolean
flag2*	boolean
int1*	int
int2*	int
int3*	int
float1*	float

* - may not be present

used - when false, action doesn't contain any additional data and should be ignored during execution, otherwise all fields are present, although some may not been used

Other fields have exactly the same meaning as fields with the same name in SET_NON_PLAYER_ENTITY_DATA.

[BREAK_BLOCK_PROGRESS] - sets progress for breaking block (visual cracks).

Name	Type
blockPos	int[3]
progress	int

blockPos - position of a block to set breaking progress (x, y, z)

progress - progress value stored in Minecraft units

[MOVEMENT] - changes entity position and head rotation

Name	Type(s)
flags	byte
posY*	short or float or double
posX*	float or double
posZ*	float or double
rotX*	short
rotY*	short
headRot*	short

* - may not be present

flags - flags packed into single byte, used to determine which fields should be present, what is their type and how should be interpreted

Flags:

Name	Mask	Value
Y_DELTA0	0b00000011 (0x03)	0b------00 (0x00)
Y_FLOAT	0b00000011 (0x03)	0b------01 (0x01)
Y_DOUBLE	0b00000011 (0x03)	0b------10 (0x02)
Y_SHORT	0b00000011 (0x03)	0b------11 (0x03)
XZ_DELTA0	0b00001100 (0x0C)	0b----00-- (0x00)
XZ_FLOAT	0b00001100 (0x0C)	0b----01-- (0x04)
XZ_DOUBLE	0b00001100 (0x0C)	0b----10-- (0x08)
invalid	0b00001100 (0x0C)	0b----11-- (0x0C)
ROT_DELTA0	0b00110000 (0x30)	0b--00---- (0x00)
ROT_HEAD_0	0b00110000 (0x30)	0b--01---- (0x10)
ROT_HEAD_EQ	0b00110000 (0x30)	0b--10---- (0x20)
ROT_HEAD_DIFF	0b00110000 (0x30)	0b--11---- (0x30)
NOT_ON_GROUND	0b01000000 (0x40)	0b-0------ (0x00)
ON_GROUND	0b01000000 (0x40)	0b-1------ (0x40)
ignored	0b10000000 (0x80)	0b0------- (0x00)
invalid	0b10000000 (0x80)	0b1------- (0x80)

Y_DELTA0 - posY is not present, y doesn't change

Y_FLOAT - posY is float and contains new y position relative to start

Y_DOUBLE - posY is double and contains new y position relative to start

Y_SHORT - posY is short and contains new absolute y position multiplied by 2 - posY = (short)(y*2)

XZ_DELTA0 - posX and posZ are not present, x and z don't change

XZ_FLOAT - posX and posZ are floats and contain new x and z positions relative to start

XZ_DOUBLE - posX and posZ are doubles and contain new x and z positions relative to start

ROT_DELTA0 - rotX, rotY and headRot are not present, rotation of body and head doesn't change

ROT_HEAD_0 - rotX and rotY are present, but headRot is not - changes body rotation and sets head rotation to 0

ROT_HEAD_EQ - rotX and rotY are present, but headRot is not - changes body rotation and sets head rotation to rotY

ROT_HEAD_DIFF - rotX, rotY and headRot are present - changes body and head rotation

NOT_ON_GROUND - player is not on ground

ON_GROUND - player is on ground

Relative positions are relative to startPos stored in recording header.

Code used to pack rotation value (in Java): (short)(((double)rot / 360.0) * (double)0x10000), where rot is float

Code used to unpack rotation value (in Java): (float)(((double)packed / (double)0x10000) * 360.0), where rot is short

Higher precision relative values (double instead of float) are chosen when value (or one of values in case of xz) is higher than max_float_pos_value setting value. Y_SHORT is chosen when it can be converted back to original value - original value multiplied by two is integer and fits on signed 16 bits. Note that this describes behavior of Motion Capture recording implementation - this isn't part of file format and is not important when reading files.

[SKIP_TICKS] - counterpart of NEXT_TICK repeated n times.

Name	Type
n	byte

[DIE] - kills main entity. It doesn't contain any additional data. It should not be called on additional entities - ENTITY_UPDATE should be used on them instead. This action doesn't end playback!

[RESPAWN] - respawns main entity after it was killed. It doesn't contain any additional data. Just like DIE, it should not be called on additional entities - ENTITY_UPDATE should be used on them instead.

[CHAT_MESSAGE] - sends chat message as played entity.

Name	Type
unused	byte
message	string

unused - unused value reserved for future uses (currently always 0), should be ignored

message - chat message encoded as Minecraft text component in JSON format

[SET_SPECTATOR] - changes entity gamemode to spectator or back to its default gamemode (depending on settings).

Name	Type
isSpectator	boolean

[DUMMY] - does nothing (no operation). It doesn't contain any additional data.

[CLOSE_CONTAINER] - causes player to close opened container. It doesn't contain any additional data.

[SET_EFFECT_PARTICLES] - causes player to close opened container. It doesn't contain any additional data.

Name	Type
particleJsonCount	packed_int
particleJsonSet	string[particleJsonCount]
ambience	boolean

particleJsonSet - array of serialized (by Minecraft) particles in JSON format

ambience - sets effect ambience - reduces count of particles (true when effect is applied by beacon)

[SET_NON_PLAYER_ENTITY_DATA] - changes some data for some entities (mainly for rideable). Replaces VEHICLE_DATA.

Name	Type	Condition
flags	byte	always present
byte1	byte	(flags & HAS_BYTE1) != 0
int1	packed_int	(flags & HAS_INT1) != 0
int2	packed_int	(flags & HAS_INT2) != 0
int3	packed_int	(flags & HAS_INT3) != 0
float1	float	(flags & HAS_FLOAT1) != 0

flags - flags packed into single byte. It has two boolean flags as entity data values and other bits determine if fields are present.

Flags:

Bit	Name	Mask
0	VAL_FLAG1	0b00000001 (0x01)
1	VAL_FLAG2	0b00000010 (0x02)
2	HAS_BYTE1	0b00000100 (0x04)
3	HAS_INT1	0b00001000 (0x08)
4	HAS_INT2	0b00010000 (0x10)
5	HAS_INT3	0b00100000 (0x20)
5	HAS_FLOAT1	0b01000000 (0x40)

VAL_FLAG1 - is dashing for Camel, has chest for AbstractChestedHorse, is right paddle turning for Boat, otherwise ignored, is in ground for AbstractArrow

VAL_FLAG2 - is baby for AgeableMob, is right paddle turning for Boat

byte1 - abstract horse shared flags (https://minecraft.wiki/w/Java_Edition_protocol/Entity_metadata#Abstract_Horse) for AbstractHorse

int1 - variant for Horse and Llama, time since last hit for Boat, shaking power for AbstractMinecart

int2 - hit direction for Boat, shaking direction for AbstractMinecart

int3 - splash timer for Boat, shaking multiplier for AbstractMinecart

float1 - damage taken for Boat

(TODO: add description for API actions)