Package org.apache.orc.impl.writer
Class UnionTreeWriter
java.lang.Object
org.apache.orc.impl.writer.TreeWriterBase
org.apache.orc.impl.writer.UnionTreeWriter
- All Implemented Interfaces:
TreeWriter
-
Nested Class Summary
Nested classes/interfaces inherited from interface org.apache.orc.impl.writer.TreeWriter
TreeWriter.Factory
-
Field Summary
Fields inherited from class org.apache.orc.impl.writer.TreeWriterBase
bloomFilter, bloomFilterEntry, bloomFilterUtf8, context, createBloomFilter, encryption, fileStatistics, id, indexStatistics, isPresent, rowIndexPosition, schema, stripeColStatistics
-
Method Summary
Modifier and TypeMethodDescriptionvoid
addStripeStatistics
(StripeStatistics[] stats) During a stripe append, we need to handle the stripe statistics.void
Create a row index entry with the previous location and the current index statistics.long
Estimate how much memory the writer is consuming excluding the streams.void
Flush the TreeWriter streamvoid
getCurrentStatistics
(ColumnStatistics[] output) Get the current file statistics for each column.long
Estimate the memory used if the file was read into Hive's Writable types.void
prepareStripe
(int stripeId) Set up for the next stripe.void
writeBatch
(org.apache.hadoop.hive.ql.exec.vector.ColumnVector vector, int offset, int length) Write the values from the given vector from offset for length elements.void
Write the FileStatistics for each column in each encryption variant.void
writeStripe
(int requiredIndexEntries) Write the stripe out to the file.Methods inherited from class org.apache.orc.impl.writer.TreeWriterBase
getRowIndex, getRowIndexEntry, getStripeStatistics, writeRootBatch
-
Method Details
-
writeBatch
public void writeBatch(org.apache.hadoop.hive.ql.exec.vector.ColumnVector vector, int offset, int length) throws IOException Description copied from class:TreeWriterBase
Write the values from the given vector from offset for length elements.- Specified by:
writeBatch
in interfaceTreeWriter
- Overrides:
writeBatch
in classTreeWriterBase
- Parameters:
vector
- the vector to write fromoffset
- the first value from the vector to writelength
- the number of values from the vector to write- Throws:
IOException
-
createRowIndexEntry
Description copied from class:TreeWriterBase
Create a row index entry with the previous location and the current index statistics. Also merges the index statistics into the file statistics before they are cleared. Finally, it records the start of the next index and ensures all of the children columns also create an entry.- Specified by:
createRowIndexEntry
in interfaceTreeWriter
- Overrides:
createRowIndexEntry
in classTreeWriterBase
- Throws:
IOException
-
writeStripe
Description copied from interface:TreeWriter
Write the stripe out to the file.- Specified by:
writeStripe
in interfaceTreeWriter
- Overrides:
writeStripe
in classTreeWriterBase
- Parameters:
requiredIndexEntries
- the number of index entries that are required. this is to check to make sure the row index is well formed.- Throws:
IOException
-
addStripeStatistics
Description copied from interface:TreeWriter
During a stripe append, we need to handle the stripe statistics.- Specified by:
addStripeStatistics
in interfaceTreeWriter
- Overrides:
addStripeStatistics
in classTreeWriterBase
- Parameters:
stats
- the statistics for the new stripe across the encryption variants- Throws:
IOException
-
estimateMemory
public long estimateMemory()Description copied from class:TreeWriterBase
Estimate how much memory the writer is consuming excluding the streams.- Specified by:
estimateMemory
in interfaceTreeWriter
- Overrides:
estimateMemory
in classTreeWriterBase
- Returns:
- the number of bytes.
-
getRawDataSize
public long getRawDataSize()Description copied from interface:TreeWriter
Estimate the memory used if the file was read into Hive's Writable types. This is used as an estimate for the query optimizer.- Returns:
- the number of bytes
-
writeFileStatistics
Description copied from interface:TreeWriter
Write the FileStatistics for each column in each encryption variant.- Specified by:
writeFileStatistics
in interfaceTreeWriter
- Overrides:
writeFileStatistics
in classTreeWriterBase
- Throws:
IOException
-
flushStreams
Description copied from interface:TreeWriter
Flush the TreeWriter stream- Specified by:
flushStreams
in interfaceTreeWriter
- Overrides:
flushStreams
in classTreeWriterBase
- Throws:
IOException
-
getCurrentStatistics
Description copied from interface:TreeWriter
Get the current file statistics for each column. If a column is encrypted, the encrypted variant statistics are used.- Specified by:
getCurrentStatistics
in interfaceTreeWriter
- Overrides:
getCurrentStatistics
in classTreeWriterBase
- Parameters:
output
- an array that is filled in with the results
-
prepareStripe
public void prepareStripe(int stripeId) Description copied from interface:TreeWriter
Set up for the next stripe.- Specified by:
prepareStripe
in interfaceTreeWriter
- Overrides:
prepareStripe
in classTreeWriterBase
- Parameters:
stripeId
- the next stripe id
-