Package io.keikai.doc.collab.lib0

Class Encoding

java.lang.Object

io.keikai.doc.collab.lib0.Encoding

public class Encoding
extends java.lang.Object

Efficient schema-less binary encoding with support for variable length encoding.

Method Summary

Modifier and Type	Method	Description
static Encoder	createEncoder()
static Uint8Array	encode(java.util.function.Consumer<Encoder> consumer)
static boolean	hasContent(Encoder encoder)	Check whether encoder is empty.
static int	length(Encoder encoder)	The current length of the encoded data.
static void	set(Encoder encoder, int pos, int num)	Write one byte at a specific position.
static void	setUint16(Encoder encoder, int pos, int num)	Write two bytes as an unsigned integer at a specific location.
static void	setUint32(Encoder encoder, int pos, int num)	Write two bytes as an unsigned integer at a specific location.
static void	setUint8(Encoder encoder, int pos, int num)	Write one byte as an unsigned Integer at a specific location.
static Uint8Array	toUint8Array(Encoder encoder)	Transform to Uint8Array.
static void	verifyLen(Encoder encoder, int len)	Verify that it is possible to write `len` bytes wtihout checking.
static void	write(Encoder encoder, int num)	Write one byte to the encoder.
static void	write(Encoder encoder, long num)	Write one byte to the encoder.
static void	writeAny(Encoder encoder, java.lang.Object data)	Encode data with efficient binary format.
static void	writeBigInt64(Encoder encoder, long num)
static void	writeBinaryEncoder(Encoder encoder, Encoder append)	Write the content of another Encoder.
static void	writeFloat32(Encoder encoder, float num)
static void	writeFloat64(Encoder encoder, double num)
static void	writeTerminatedString(Encoder encoder, java.lang.String str)	Write a string terminated by a special byte sequence.
static void	writeTerminatedUint8Array(Encoder encoder, Uint8Array buf)	Write a terminating Uint8Array.
static void	writeUint16(Encoder encoder, int num)	Write two bytes as an unsigned integer.
static void	writeUint32(Encoder encoder, int num)	Write two bytes as an unsigned integer
static void	writeUint32BigEndian(Encoder encoder, int num)	Write two bytes as an unsigned integer in big endian order.
static void	writeUint8(Encoder encoder, int num)	Write one byte as an unsigned integer.
static void	writeUint8Array(Encoder encoder, Uint8Array uint8Array)	Append fixed-length Uint8Array to the encoder.
static void	writeVarInt(Encoder encoder, int num)	Write a variable length integer.
static void	writeVarInt(Encoder encoder, long num)	Write a variable length long.
static void	writeVarString(Encoder encoder, java.lang.String str)	Write a variable length string.
static void	writeVarUint(Encoder encoder, int num)	Write a variable length unsigned integer.
static void	writeVarUint(Encoder encoder, long num)	Write a variable length unsigned long.
static void	writeVarUint8Array(Encoder encoder, Uint8Array uint8Array)	Append an Uint8Array to Encoder.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

createEncoder

public static Encoder createEncoder()

encode

public static Uint8Array encode(java.util.function.Consumer<Encoder> consumer)

length

public static int length(Encoder encoder)

The current length of the encoded data.

Parameters:

encoder -

Returns:

hasContent

public static boolean hasContent(Encoder encoder)

Check whether encoder is empty.

Parameters:

encoder -

Returns:

toUint8Array

public static Uint8Array toUint8Array(Encoder encoder)

Transform to Uint8Array.

Parameters:

encoder -

Returns:

verifyLen

public static void verifyLen(Encoder encoder,
                             int len)

Verify that it is possible to write `len` bytes wtihout checking. If necessary, a new Buffer with the required length is attached.

Parameters:

encoder -

len -

write

public static void write(Encoder encoder,
                         int num)

Write one byte to the encoder.

Parameters:

encoder -

num -

write

public static void write(Encoder encoder,
                         long num)

Write one byte to the encoder.

Parameters:

encoder -

num -

set

public static void set(Encoder encoder,
                       int pos,
                       int num)

Write one byte at a specific position. Position must already be written (i.e. encoder.length > pos)

Parameters:

encoder -

pos -

num -

writeUint8

public static void writeUint8(Encoder encoder,
                              int num)

Write one byte as an unsigned integer.

Parameters:

encoder -

num -

setUint8

public static void setUint8(Encoder encoder,
                            int pos,
                            int num)

Write one byte as an unsigned Integer at a specific location.

Parameters:

encoder -

pos -

num -

writeUint16

public static void writeUint16(Encoder encoder,
                               int num)

Write two bytes as an unsigned integer.

Parameters:

encoder -

num -

setUint16

public static void setUint16(Encoder encoder,
                             int pos,
                             int num)

Write two bytes as an unsigned integer at a specific location.

Parameters:

encoder -

pos -

num -

writeUint32

public static void writeUint32(Encoder encoder,
                               int num)

Write two bytes as an unsigned integer

Parameters:

encoder -

num -

writeUint32BigEndian

public static void writeUint32BigEndian(Encoder encoder,
                                        int num)

Write two bytes as an unsigned integer in big endian order. (most significant byte first)

Parameters:

encoder -

num -

setUint32

public static void setUint32(Encoder encoder,
                             int pos,
                             int num)

Write two bytes as an unsigned integer at a specific location.

Parameters:

encoder -

pos -

num -

writeVarUint

public static void writeVarUint(Encoder encoder,
                                int num)

Write a variable length unsigned integer. Acceptable max integer is 2^31.

Parameters:

encoder -

num -

writeVarUint

public static void writeVarUint(Encoder encoder,
                                long num)

Write a variable length unsigned long.

Parameters:

encoder -

num -

writeVarInt

public static void writeVarInt(Encoder encoder,
                               int num)

Write a variable length integer. Acceptable max integer is 2^31.

Parameters:

encoder -

num -

writeVarInt

public static void writeVarInt(Encoder encoder,
                               long num)

Write a variable length long.

Parameters:

encoder -

num -

writeVarString

public static void writeVarString(Encoder encoder,
                                  java.lang.String str)

Write a variable length string.

Parameters:

encoder -

str -

writeTerminatedString

public static void writeTerminatedString(Encoder encoder,
                                         java.lang.String str)

Write a string terminated by a special byte sequence. This is not very performant and is generally discouraged. However, the resulting byte arrays are lexiographically ordered which makes this a nice feature for databases.

The string will be encoded using utf8 and then terminated and escaped using writeTerminatingUint8Array.

Parameters:

encoder -

str -

writeTerminatedUint8Array

public static void writeTerminatedUint8Array(Encoder encoder,
                                             Uint8Array buf)

Write a terminating Uint8Array. Note that this is not performant and is generally discouraged. There are few situations when this is needed.

We use 0x0 as a terminating character. 0x1 serves as an escape character for 0x0 and 0x1.

Example: [0,1,2] is encoded to [1,0,1,1,2,0]. 0x0, and 0x1 needed to be escaped using 0x1. Then the result is terminated using the 0x0 character.

This is basically how many systems implement null terminated strings. However, we use an escape character 0x1 to avoid issues and potenial attacks on our database (if this is used as a key encoder for NoSql databases).

Parameters:

encoder -

buf -

writeBinaryEncoder

public static void writeBinaryEncoder(Encoder encoder,
                                      Encoder append)

Write the content of another Encoder.

Parameters:

encoder -

append -

writeUint8Array

public static void writeUint8Array(Encoder encoder,
                                   Uint8Array uint8Array)

Append fixed-length Uint8Array to the encoder.

Parameters:

encoder -

uint8Array -

writeVarUint8Array

public static void writeVarUint8Array(Encoder encoder,
                                      Uint8Array uint8Array)

Append an Uint8Array to Encoder.

Parameters:

encoder -

uint8Array -

writeFloat32

public static void writeFloat32(Encoder encoder,
                                float num)

writeFloat64

public static void writeFloat64(Encoder encoder,
                                double num)

writeBigInt64

public static void writeBigInt64(Encoder encoder,
                                 long num)

writeAny

public static void writeAny(Encoder encoder,
                            java.lang.Object data)

Encode data with efficient binary format.

Differences to JSON: • Transforms data to a binary format (not to a string) • Encodes undefined, NaN, and ArrayBuffer (these can't be represented in JSON) • Numbers are efficiently encoded either as a variable length integer, as a 32 bit float, as a 64 bit float, or as a 64 bit bigint.

Encoding table:

| Data Type | Prefix | Encoding Method | Comment | | ------------------- | -------- | ------------------ | ------- | | undefined | 127 | | Functions, symbol, and everything that cannot be identified is encoded as undefined | | null | 126 | | | | integer | 125 | writeVarInt | Only encodes 32 bit signed integers | | float32 | 124 | writeFloat32 | | | float64 | 123 | writeFloat64 | | | bigint | 122 | writeBigInt64 | | | boolean (false) | 121 | | True and false are different data types so we save the following byte | | boolean (true) | 120 | | - 0b01111000 so the last bit determines whether true or false | | string | 119 | writeVarString | | | Map | 118 | custom | Writes {length} then {length} key-value pairs | | array | 117 | custom | Writes {length} then {length} json values | | Uint8Array | 116 | writeVarUint8Array | We use Uint8Array for any kind of binary data |

Reasons for the decreasing prefix: We need the first bit for extendability (later we may want to encode the prefix with writeVarUint). The remaining 7 bits are divided as follows: [0-30] the beginning of the data range is used for custom purposes (defined by the function that uses this library) [31-127] the end of the data range is used for data encoding by lib0/encoding.js

Parameters:

encoder -

data - the type might be {null|number|bigint|boolean|string|Map|Array|Uint8Array}