Package org.apache.orc.impl.writer

Class EncryptionTreeWriter

java.lang.Object

org.apache.orc.impl.writer.EncryptionTreeWriter

All Implemented Interfaces:

TreeWriter

public class EncryptionTreeWriter
extends Object
implements TreeWriter

TreeWriter that handles column encryption. We create a TreeWriter for each of the alternatives with an WriterContext that creates encrypted streams.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.orc.impl.writer.TreeWriter

TreeWriter.Factory

Method Summary

Modifier and Type	Method	Description
void	addStripeStatistics(StripeStatistics[] stripeStatistics)	During a stripe append, we need to handle the stripe statistics.
void	createRowIndexEntry()	Create a row index entry at the current point in the stripe.
long	estimateMemory()	Estimate the memory currently used to buffer the stripe.
void	flushStreams()	Flush the TreeWriter stream
void	getCurrentStatistics(ColumnStatistics[] output)	Get the current file statistics for each column.
long	getRawDataSize()	Estimate the memory used if the file was read into Hive's Writable types.
void	prepareStripe(int stripeId)	Set up for the next stripe.
void	writeBatch(org.apache.hadoop.hive.ql.exec.vector.ColumnVector vector, int offset, int length)	Write a ColumnVector to the file.
void	writeFileStatistics()	Write the FileStatistics for each column in each encryption variant.
void	writeRootBatch(org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch batch, int offset, int length)	Write a VectorizedRowBatch to the file.
void	writeStripe(int requiredIndexEntries)	Write the stripe out to the file.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

writeRootBatch

public void writeRootBatch(org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch batch,
 int offset,
 int length)
                    throws IOException

Description copied from interface: TreeWriter

Write a VectorizedRowBatch to the file. This is called by the WriterImplV2 at the top level.

Specified by:

writeRootBatch in interface TreeWriter

Parameters:

batch - the list of all of the columns

offset - the first row from the batch to write

length - the number of rows to write

Throws:

IOException

writeBatch

public void writeBatch(org.apache.hadoop.hive.ql.exec.vector.ColumnVector vector,
 int offset,
 int length)
                throws IOException

Description copied from interface: TreeWriter

Write a ColumnVector to the file. This is called recursively by writeRootBatch.

Specified by:

writeBatch in interface TreeWriter

Parameters:

vector - the data to write

offset - the first value offset to write.

length - the number of values to write

Throws:

IOException

createRowIndexEntry

public void createRowIndexEntry()
                         throws IOException

Description copied from interface: TreeWriter

Create a row index entry at the current point in the stripe.

Specified by:

createRowIndexEntry in interface TreeWriter

Throws:

IOException

flushStreams

public void flushStreams()
                  throws IOException

Description copied from interface: TreeWriter

Flush the TreeWriter stream

Specified by:

flushStreams in interface TreeWriter

Throws:

IOException

writeStripe

public void writeStripe(int requiredIndexEntries)
                 throws IOException

Description copied from interface: TreeWriter

Write the stripe out to the file.

Specified by:

writeStripe in interface TreeWriter

Parameters:

requiredIndexEntries - the number of index entries that are required. this is to check to make sure the row index is well formed.

Throws:

IOException

addStripeStatistics

public void addStripeStatistics(StripeStatistics[] stripeStatistics)
                         throws IOException

Description copied from interface: TreeWriter

During a stripe append, we need to handle the stripe statistics.

Specified by:

addStripeStatistics in interface TreeWriter

Parameters:

stripeStatistics - the statistics for the new stripe across the encryption variants

Throws:

IOException

estimateMemory

public long estimateMemory()

Description copied from interface: TreeWriter

Estimate the memory currently used to buffer the stripe.

Specified by:

estimateMemory in interface TreeWriter

Returns:

the number of bytes

getRawDataSize

public long getRawDataSize()

Description copied from interface: TreeWriter

Estimate the memory used if the file was read into Hive's Writable types. This is used as an estimate for the query optimizer.

Specified by:

getRawDataSize in interface TreeWriter

Returns:

the number of bytes

prepareStripe

public void prepareStripe(int stripeId)

Description copied from interface: TreeWriter

Set up for the next stripe.

Specified by:

prepareStripe in interface TreeWriter

Parameters:

stripeId - the next stripe id

writeFileStatistics

public void writeFileStatistics()
                         throws IOException

Description copied from interface: TreeWriter

Write the FileStatistics for each column in each encryption variant.

Specified by:

writeFileStatistics in interface TreeWriter

Throws:

IOException

getCurrentStatistics

public void getCurrentStatistics(ColumnStatistics[] output)

Description copied from interface: TreeWriter

Get the current file statistics for each column. If a column is encrypted, the encrypted variant statistics are used.

Specified by:

getCurrentStatistics in interface TreeWriter

Parameters:

output - an array that is filled in with the results