Package io.keikai.doc.collab.lib0

Class Encoding

java.lang.Object
io.keikai.doc.collab.lib0.Encoding
public class Encoding extends Object
Efficient schema-less binary encoding with support for variable length encoding.

  • Method Details

    • createEncoder

      public static Encoder createEncoder()

    • encode

      public static Uint8Array encode(Consumer<Encoder> consumer)

    • length

      public static int length(Encoder encoder)
      The current length of the encoded data.
      Parameters:
      encoder -
      Returns:

    • hasContent

      public static boolean hasContent(Encoder encoder)
      Check whether encoder is empty.
      Parameters:
      encoder -
      Returns:

    • toUint8Array

      public static Uint8Array toUint8Array(Encoder encoder)
      Transform to Uint8Array.
      Parameters:
      encoder -
      Returns:

    • verifyLen

      public static void verifyLen(Encoder encoder, int len)
      Verify that it is possible to write `len` bytes wtihout checking. If necessary, a new Buffer with the required length is attached.
      Parameters:
      encoder -
      len -

    • write

      public static void write(Encoder encoder, int num)
      Write one byte to the encoder.
      Parameters:
      encoder -
      num -

    • write

      public static void write(Encoder encoder, long num)
      Write one byte to the encoder.
      Parameters:
      encoder -
      num -

    • set

      public static void set(Encoder encoder, int pos, int num)
      Write one byte at a specific position. Position must already be written (i.e. encoder.length > pos)
      Parameters:
      encoder -
      pos -
      num -

    • writeUint8

      public static void writeUint8(Encoder encoder, int num)
      Write one byte as an unsigned integer.
      Parameters:
      encoder -
      num -

    • setUint8

      public static void setUint8(Encoder encoder, int pos, int num)
      Write one byte as an unsigned Integer at a specific location.
      Parameters:
      encoder -
      pos -
      num -

    • writeUint16

      public static void writeUint16(Encoder encoder, int num)
      Write two bytes as an unsigned integer.
      Parameters:
      encoder -
      num -

    • setUint16

      public static void setUint16(Encoder encoder, int pos, int num)
      Write two bytes as an unsigned integer at a specific location.
      Parameters:
      encoder -
      pos -
      num -

    • writeUint32

      public static void writeUint32(Encoder encoder, int num)
      Write two bytes as an unsigned integer
      Parameters:
      encoder -
      num -

    • writeUint32BigEndian

      public static void writeUint32BigEndian(Encoder encoder, int num)
      Write two bytes as an unsigned integer in big endian order. (most significant byte first)
      Parameters:
      encoder -
      num -

    • setUint32

      public static void setUint32(Encoder encoder, int pos, int num)
      Write two bytes as an unsigned integer at a specific location.
      Parameters:
      encoder -
      pos -
      num -

    • writeVarUint

      public static void writeVarUint(Encoder encoder, int num)
      Write a variable length unsigned integer. Acceptable max integer is 2^31.
      Parameters:
      encoder -
      num -

    • writeVarUint

      public static void writeVarUint(Encoder encoder, long num)
      Write a variable length unsigned long.
      Parameters:
      encoder -
      num -

    • writeVarInt

      public static void writeVarInt(Encoder encoder, int num)
      Write a variable length integer. Acceptable max integer is 2^31.
      Parameters:
      encoder -
      num -

    • writeVarInt

      public static void writeVarInt(Encoder encoder, long num)
      Write a variable length long.
      Parameters:
      encoder -
      num -

    • writeVarString

      public static void writeVarString(Encoder encoder, String str)
      Write a variable length string.
      Parameters:
      encoder -
      str -

    • writeTerminatedString

      public static void writeTerminatedString(Encoder encoder, String str)
      Write a string terminated by a special byte sequence. This is not very performant and is generally discouraged. However, the resulting byte arrays are lexiographically ordered which makes this a nice feature for databases.

      The string will be encoded using utf8 and then terminated and escaped using writeTerminatingUint8Array.

      Parameters:
      encoder -
      str -

    • writeTerminatedUint8Array

      public static void writeTerminatedUint8Array(Encoder encoder, Uint8Array buf)
      Write a terminating Uint8Array. Note that this is not performant and is generally discouraged. There are few situations when this is needed.

      We use 0x0 as a terminating character. 0x1 serves as an escape character for 0x0 and 0x1.

      Example: [0,1,2] is encoded to [1,0,1,1,2,0]. 0x0, and 0x1 needed to be escaped using 0x1. Then the result is terminated using the 0x0 character.

      This is basically how many systems implement null terminated strings. However, we use an escape character 0x1 to avoid issues and potenial attacks on our database (if this is used as a key encoder for NoSql databases).

      Parameters:
      encoder -
      buf -

    • writeBinaryEncoder

      public static void writeBinaryEncoder(Encoder encoder, Encoder append)
      Write the content of another Encoder.
      Parameters:
      encoder -
      append -

    • writeUint8Array

      public static void writeUint8Array(Encoder encoder, Uint8Array uint8Array)
      Append fixed-length Uint8Array to the encoder.
      Parameters:
      encoder -
      uint8Array -

    • writeVarUint8Array

      public static void writeVarUint8Array(Encoder encoder, Uint8Array uint8Array)
      Append an Uint8Array to Encoder.
      Parameters:
      encoder -
      uint8Array -

    • writeFloat32

      public static void writeFloat32(Encoder encoder, float num)

    • writeFloat64

      public static void writeFloat64(Encoder encoder, double num)

    • writeBigInt64

      public static void writeBigInt64(Encoder encoder, long num)

    • writeAny

      public static void writeAny(Encoder encoder, Object data)
      Encode data with efficient binary format.

      Differences to JSON: • Transforms data to a binary format (not to a string) • Encodes undefined, NaN, and ArrayBuffer (these can't be represented in JSON) • Numbers are efficiently encoded either as a variable length integer, as a 32 bit float, as a 64 bit float, or as a 64 bit bigint.

      Encoding table:

      | Data Type | Prefix | Encoding Method | Comment | | ------------------- | -------- | ------------------ | ------- | | undefined | 127 | | Functions, symbol, and everything that cannot be identified is encoded as undefined | | null | 126 | | | | integer | 125 | writeVarInt | Only encodes 32 bit signed integers | | float32 | 124 | writeFloat32 | | | float64 | 123 | writeFloat64 | | | bigint | 122 | writeBigInt64 | | | boolean (false) | 121 | | True and false are different data types so we save the following byte | | boolean (true) | 120 | | - 0b01111000 so the last bit determines whether true or false | | string | 119 | writeVarString | | | Map | 118 | custom | Writes {length} then {length} key-value pairs | | array | 117 | custom | Writes {length} then {length} json values | | Uint8Array | 116 | writeVarUint8Array | We use Uint8Array for any kind of binary data |

      Reasons for the decreasing prefix: We need the first bit for extendability (later we may want to encode the prefix with writeVarUint). The remaining 7 bits are divided as follows: [0-30] the beginning of the data range is used for custom purposes (defined by the function that uses this library) [31-127] the end of the data range is used for data encoding by lib0/encoding.js

      Parameters:
      encoder -
      data - the type might be {null|number|bigint|boolean|string|Map|Array|Uint8Array}