Package org.apache.orc

Interface PhysicalWriter

All Known Implementing Classes:
PhysicalFsWriter
public interface PhysicalWriter
This interface separates the physical layout of ORC files from the higher level details.

This API is limited to being used by LLAP.

  • Method Details

    • writeHeader

      void writeHeader() throws IOException
      Writes the header of the file, which consists of the magic "ORC" bytes.
      Throws:
      IOException

    • createDataStream

      PhysicalWriter.OutputReceiver createDataStream(StreamName name) throws IOException
      Create an OutputReceiver for the given name.
      Parameters:
      name - the name of the stream
      Throws:
      IOException

    • writeIndex

      void writeIndex(StreamName name, OrcProto.RowIndex.Builder index) throws IOException
      Write an index in the given stream name.
      Parameters:
      name - the name of the stream
      index - the bloom filter to write
      Throws:
      IOException

    • writeBloomFilter

      void writeBloomFilter(StreamName name, OrcProto.BloomFilterIndex.Builder bloom) throws IOException
      Write a bloom filter index in the given stream name.
      Parameters:
      name - the name of the stream
      bloom - the bloom filter to write
      Throws:
      IOException

    • finalizeStripe

      void finalizeStripe(OrcProto.StripeFooter.Builder footer, OrcProto.StripeInformation.Builder dirEntry) throws IOException
      Flushes the data in all the streams, spills them to disk, write out stripe footer.
      Parameters:
      footer - Stripe footer to be updated with relevant data and written out.
      dirEntry - File metadata entry for the stripe, to be updated with relevant data.
      Throws:
      IOException

    • writeStatistics

      void writeStatistics(StreamName name, OrcProto.ColumnStatistics.Builder statistics) throws IOException
      Write a stripe or file statistics to the file.
      Parameters:
      name - the name of the stream
      statistics - the statistics to write
      Throws:
      IOException

    • writeFileMetadata

      void writeFileMetadata(OrcProto.Metadata.Builder builder) throws IOException
      Writes out the file metadata.
      Parameters:
      builder - Metadata builder to finalize and write.
      Throws:
      IOException

    • writeFileFooter

      void writeFileFooter(OrcProto.Footer.Builder builder) throws IOException
      Writes out the file footer.
      Parameters:
      builder - Footer builder to finalize and write.
      Throws:
      IOException

    • writePostScript

      long writePostScript(OrcProto.PostScript.Builder builder) throws IOException
      Writes out the postscript (including the size byte if needed).
      Parameters:
      builder - Postscript builder to finalize and write.
      Throws:
      IOException

    • close

      void close() throws IOException
      Closes the writer.
      Throws:
      IOException

    • flush

      void flush() throws IOException
      Flushes the writer so that readers can see the preceding postscripts.
      Throws:
      IOException

    • appendRawStripe

      void appendRawStripe(ByteBuffer stripe, OrcProto.StripeInformation.Builder dirEntry) throws IOException
      Appends raw stripe data (e.g. for file merger).
      Parameters:
      stripe - Stripe data buffer.
      dirEntry - File metadata entry for the stripe, to be updated with relevant data.
      Throws:
      IOException

    • getFileBytes

      long getFileBytes(int column, WriterEncryptionVariant variant)
      Get the number of bytes for a file in a given column.
      Parameters:
      column - column from which to get file size
      variant - the encryption variant to check
      Returns:
      number of bytes for the given column

    • getStreamOptions

      StreamOptions getStreamOptions()
      Get the unencrypted stream options for this file. This class needs the stream options to write the indexes and footers. Additionally, the LLAP CacheWriter wants to disable the generic compression.