Package org.apache.orc.impl

Class SerializationUtils

java.lang.Object
org.apache.orc.impl.SerializationUtils
public final class SerializationUtils extends Object

  • Constructor Details

    • SerializationUtils

      public SerializationUtils()

  • Method Details

    • writeVulong

      public void writeVulong(OutputStream output, long value) throws IOException
      Throws:
      IOException

    • writeVslong

      public void writeVslong(OutputStream output, long value) throws IOException
      Throws:
      IOException

    • readVulong

      public static long readVulong(InputStream in) throws IOException
      Throws:
      IOException

    • readVslong

      public static long readVslong(InputStream in) throws IOException
      Throws:
      IOException

    • readFloat

      public float readFloat(InputStream in) throws IOException
      Throws:
      IOException

    • skipFloat

      public void skipFloat(InputStream in, int numOfFloats) throws IOException
      Throws:
      IOException

    • writeFloat

      public void writeFloat(OutputStream output, float value) throws IOException
      Throws:
      IOException

    • readDouble

      public double readDouble(InputStream in) throws IOException
      Throws:
      IOException

    • readLongLE

      public long readLongLE(InputStream in) throws IOException
      Throws:
      IOException

    • skipDouble

      public void skipDouble(InputStream in, int numOfDoubles) throws IOException
      Throws:
      IOException

    • writeDouble

      public void writeDouble(OutputStream output, double value) throws IOException
      Throws:
      IOException

    • writeBigInteger

      public static void writeBigInteger(OutputStream output, BigInteger value) throws IOException
      Write the arbitrarily sized signed BigInteger in vint format. Signed integers are encoded using the low bit as the sign bit using zigzag encoding. Each byte uses the low 7 bits for data and the high bit for stop/continue. Bytes are stored LSB first.
      Parameters:
      output - the stream to write to
      value - the value to output
      Throws:
      IOException

    • readBigInteger

      public static BigInteger readBigInteger(InputStream input) throws IOException
      Read the signed arbitrary sized BigInteger BigInteger in vint format
      Parameters:
      input - the stream to read from
      Returns:
      the read BigInteger
      Throws:
      IOException

    • findClosestNumBits

      public int findClosestNumBits(long value)
      Count the number of bits required to encode the given value
      Parameters:
      value -
      Returns:
      bits required to store value

    • zigzagEncode

      public long zigzagEncode(long val)
      zigzag encode the given value
      Parameters:
      val -
      Returns:
      zigzag encoded value

    • zigzagDecode

      public long zigzagDecode(long val)
      zigzag decode the given value
      Parameters:
      val -
      Returns:
      zizag decoded value

    • percentileBits

      public int percentileBits(long[] data, int offset, int length, double p)
      Compute the bits required to represent pth percentile value
      Parameters:
      data - - array
      p - - percentile value (>=0.0 to <=1.0)
      Returns:
      pth percentile bits

    • bytesToLongBE

      public long bytesToLongBE(InStream input, int n) throws IOException
      Read n bytes in big endian order and convert to long
      Returns:
      long value
      Throws:
      IOException

    • getClosestFixedBits

      public int getClosestFixedBits(int n)
      For a given fixed bit this function will return the closest available fixed bit
      Parameters:
      n -
      Returns:
      closest valid fixed bit

    • getClosestAlignedFixedBits

      public int getClosestAlignedFixedBits(int n)

    • encodeBitWidth

      public int encodeBitWidth(int n)
      Finds the closest available fixed bit width match and returns its encoded value (ordinal).
      Parameters:
      n - fixed bit width to encode
      Returns:
      encoded fixed bit width

    • decodeBitWidth

      public static int decodeBitWidth(int n)
      Decodes the ordinal fixed bit value to actual fixed bit width value
      Parameters:
      n - - encoded fixed bit width
      Returns:
      decoded fixed bit width

    • writeInts

      public void writeInts(long[] input, int offset, int len, int bitSize, OutputStream output) throws IOException
      Bitpack and write the input values to underlying output stream
      Parameters:
      input - - values to write
      offset - - offset
      len - - length
      bitSize - - bit width
      output - - output stream
      Throws:
      IOException

    • readInts

      public void readInts(long[] buffer, int offset, int len, int bitSize, InStream input) throws IOException
      Read bitpacked integers from input stream
      Parameters:
      buffer - - input buffer
      offset - - offset
      len - - length
      bitSize - - bit width
      input - - input stream
      Throws:
      IOException

    • isSafeSubtract

      public boolean isSafeSubtract(long left, long right)

    • convertFromUtc

      public static double convertFromUtc(TimeZone local, double time)
      Convert a UTC time to a local timezone
      Parameters:
      local - the local timezone
      time - the number of seconds since 1970
      Returns:
      the converted timestamp

    • convertFromUtc

      public static long convertFromUtc(TimeZone local, long time)

    • convertToUtc

      public static long convertToUtc(TimeZone local, long time)

    • getCustomizedCodec

      public static StreamOptions getCustomizedCodec(StreamOptions base, OrcFile.CompressionStrategy strategy, OrcProto.Stream.Kind kind)
      Get the stream options with the compression tuned for the particular kind of stream.
      Parameters:
      base - the original options
      strategy - the compression strategy
      kind - the stream kind
      Returns:
      the tuned options or the original if it is the same

    • convertBetweenTimezones

      public static long convertBetweenTimezones(TimeZone writer, TimeZone reader, long millis)
      Find the relative offset when moving between timezones at a particular point in time. This is a function of ORC v0 and v1 writing timestamps relative to the local timezone. Therefore, when we read, we need to convert from the writer's timezone to the reader's timezone.
      Parameters:
      writer - the timezone we are moving from
      reader - the timezone we are moving to
      millis - the point in time
      Returns:
      the change in milliseconds

    • bytesVectorToString

      public static String bytesVectorToString(org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector vector, int elementNum)
      Convert a bytes vector element into a String.
      Parameters:
      vector - the vector to use
      elementNum - the element number to stringify
      Returns:
      a string or null if the value was null

    • parseDateFromString

      public static Date parseDateFromString(String string)
      Parse a date from a string.
      Parameters:
      string - the date to parse (YYYY-MM-DD)
      Returns:
      the Date parsed, or null if there was a parse error.