Package org.apache.lucene.store
Class ByteBuffersDataOutput
- java.lang.Object
-
- org.apache.lucene.store.DataOutput
-
- org.apache.lucene.store.ByteBuffersDataOutput
-
- All Implemented Interfaces:
Accountable
public final class ByteBuffersDataOutput extends DataOutput implements Accountable
ADataOutput
storing data in a list ofByteBuffer
s.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
ByteBuffersDataOutput.ByteBufferRecycler
An implementation of aByteBuffer
allocation and recycling policy.
-
Field Summary
Fields Modifier and Type Field Description static java.util.function.IntFunction<java.nio.ByteBuffer>
ALLOCATE_BB_ON_HEAP
private java.util.function.IntFunction<java.nio.ByteBuffer>
blockAllocate
ByteBuffer
supplier.private int
blockBits
Current block size:2^bits
.private java.util.function.Consumer<java.nio.ByteBuffer>
blockReuse
ByteBuffer
recycler onreset()
.private java.util.ArrayDeque<java.nio.ByteBuffer>
blocks
Blocks storing data.private java.nio.ByteBuffer
currentBlock
The current-or-next write block.static int
DEFAULT_MAX_BITS_PER_BLOCK
static int
DEFAULT_MIN_BITS_PER_BLOCK
private static java.nio.ByteBuffer
EMPTY
private static byte[]
EMPTY_BYTE_ARRAY
private static long
HALF_SHIFT
(package private) static int
MAX_BLOCKS_BEFORE_BLOCK_EXPANSION
Maximum number of blocks at the currentblockBits
block size before we increase the block size (and thus decrease the number of blocks).private int
maxBitsPerBlock
Maximum block size:2^bits
.static java.util.function.Consumer<java.nio.ByteBuffer>
NO_REUSE
A singleton instance of "no-reuse" buffer strategy.private static int
SURROGATE_OFFSET
-
Constructor Summary
Constructors Constructor Description ByteBuffersDataOutput()
ByteBuffersDataOutput(int minBitsPerBlock, int maxBitsPerBlock, java.util.function.IntFunction<java.nio.ByteBuffer> blockAllocate, java.util.function.Consumer<java.nio.ByteBuffer> blockReuse)
ByteBuffersDataOutput(long expectedSize)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description private void
appendBlock()
private int
blockSize()
private static int
computeBlockSizeBitsFor(long bytes)
void
copyTo(DataOutput output)
Copy the current content of this object into anotherDataOutput
.static ByteBuffersDataOutput
newResettableInstance()
long
ramBytesUsed()
Return the memory usage of this object in bytes.void
reset()
This method resets this object to a clean (zero-size) state and publishes any currently allocated buffers for reuse to the reuse strategy provided in the constructor.private void
rewriteToBlockSize(int targetBlockBits)
long
size()
byte[]
toArrayCopy()
Return a contiguous array with the current content written to the output.java.util.ArrayList<java.nio.ByteBuffer>
toBufferList()
Return a list of read-only view ofByteBuffer
blocks over the current content written to the output.ByteBuffersDataInput
toDataInput()
Return aByteBuffersDataInput
for the set of current buffers (toBufferList()
).java.lang.String
toString()
java.util.ArrayList<java.nio.ByteBuffer>
toWriteableBufferList()
Returns a list of writeable blocks over the (source) content buffers.private static int
UTF16toUTF8(java.lang.CharSequence s, int offset, int length, byte[] buf, java.util.function.IntConsumer bufferFlusher)
A consumer-based UTF16-UTF8 encoder (writes the input string in smaller buffers.).void
writeByte(byte b)
Writes a single byte.void
writeBytes(byte[] b)
void
writeBytes(byte[] b, int length)
Writes an array of bytes.void
writeBytes(byte[] src, int offset, int length)
Writes an array of bytes.void
writeBytes(java.nio.ByteBuffer buffer)
void
writeInt(int v)
Writes an int as four bytes.void
writeLong(long v)
Writes a long as eight bytes.void
writeMapOfStrings(java.util.Map<java.lang.String,java.lang.String> map)
Writes a String map.void
writeSetOfStrings(java.util.Set<java.lang.String> set)
Writes a String set.void
writeShort(short v)
Writes a short as two bytes.void
writeString(java.lang.String v)
Writes a string.-
Methods inherited from class org.apache.lucene.store.DataOutput
copyBytes, writeVInt, writeVLong, writeZInt, writeZLong
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
-
Methods inherited from interface org.apache.lucene.util.Accountable
getChildResources
-
-
-
-
Field Detail
-
EMPTY
private static final java.nio.ByteBuffer EMPTY
-
EMPTY_BYTE_ARRAY
private static final byte[] EMPTY_BYTE_ARRAY
-
ALLOCATE_BB_ON_HEAP
public static final java.util.function.IntFunction<java.nio.ByteBuffer> ALLOCATE_BB_ON_HEAP
-
NO_REUSE
public static final java.util.function.Consumer<java.nio.ByteBuffer> NO_REUSE
A singleton instance of "no-reuse" buffer strategy.
-
DEFAULT_MIN_BITS_PER_BLOCK
public static final int DEFAULT_MIN_BITS_PER_BLOCK
- See Also:
- Constant Field Values
-
DEFAULT_MAX_BITS_PER_BLOCK
public static final int DEFAULT_MAX_BITS_PER_BLOCK
- See Also:
- Constant Field Values
-
MAX_BLOCKS_BEFORE_BLOCK_EXPANSION
static final int MAX_BLOCKS_BEFORE_BLOCK_EXPANSION
Maximum number of blocks at the currentblockBits
block size before we increase the block size (and thus decrease the number of blocks).- See Also:
- Constant Field Values
-
maxBitsPerBlock
private final int maxBitsPerBlock
Maximum block size:2^bits
.
-
blockAllocate
private final java.util.function.IntFunction<java.nio.ByteBuffer> blockAllocate
ByteBuffer
supplier.
-
blockReuse
private final java.util.function.Consumer<java.nio.ByteBuffer> blockReuse
ByteBuffer
recycler onreset()
.
-
blockBits
private int blockBits
Current block size:2^bits
.
-
blocks
private final java.util.ArrayDeque<java.nio.ByteBuffer> blocks
Blocks storing data.
-
currentBlock
private java.nio.ByteBuffer currentBlock
The current-or-next write block.
-
HALF_SHIFT
private static final long HALF_SHIFT
- See Also:
- Constant Field Values
-
SURROGATE_OFFSET
private static final int SURROGATE_OFFSET
- See Also:
- Constant Field Values
-
-
Constructor Detail
-
ByteBuffersDataOutput
public ByteBuffersDataOutput(long expectedSize)
-
ByteBuffersDataOutput
public ByteBuffersDataOutput()
-
ByteBuffersDataOutput
public ByteBuffersDataOutput(int minBitsPerBlock, int maxBitsPerBlock, java.util.function.IntFunction<java.nio.ByteBuffer> blockAllocate, java.util.function.Consumer<java.nio.ByteBuffer> blockReuse)
-
-
Method Detail
-
writeByte
public void writeByte(byte b)
Description copied from class:DataOutput
Writes a single byte.The most primitive data type is an eight-bit byte. Files are accessed as sequences of bytes. All other data types are defined as sequences of bytes, so file formats are byte-order independent.
- Specified by:
writeByte
in classDataOutput
- See Also:
DataInput.readByte()
-
writeBytes
public void writeBytes(byte[] src, int offset, int length)
Description copied from class:DataOutput
Writes an array of bytes.- Specified by:
writeBytes
in classDataOutput
- Parameters:
src
- the bytes to writeoffset
- the offset in the byte arraylength
- the number of bytes to write- See Also:
DataInput.readBytes(byte[],int,int)
-
writeBytes
public void writeBytes(byte[] b, int length)
Description copied from class:DataOutput
Writes an array of bytes.- Overrides:
writeBytes
in classDataOutput
- Parameters:
b
- the bytes to writelength
- the number of bytes to write- See Also:
DataInput.readBytes(byte[],int,int)
-
writeBytes
public void writeBytes(byte[] b)
-
writeBytes
public void writeBytes(java.nio.ByteBuffer buffer)
-
toBufferList
public java.util.ArrayList<java.nio.ByteBuffer> toBufferList()
Return a list of read-only view ofByteBuffer
blocks over the current content written to the output.
-
toWriteableBufferList
public java.util.ArrayList<java.nio.ByteBuffer> toWriteableBufferList()
Returns a list of writeable blocks over the (source) content buffers. This method returns the raw content of source buffers that may change over the lifetime of this object (blocks can be recycled or discarded, for example). Most applications should favor callingtoBufferList()
which returns a read-only view over the content of the source buffers. The difference betweentoBufferList()
andtoWriteableBufferList()
is that read-only view of source buffers will always returnfalse
fromByteBuffer.hasArray()
(which sometimes may be required to avoid double copying).
-
toDataInput
public ByteBuffersDataInput toDataInput()
Return aByteBuffersDataInput
for the set of current buffers (toBufferList()
).
-
toArrayCopy
public byte[] toArrayCopy()
Return a contiguous array with the current content written to the output. The returned array is always a copy (can be mutated). If thesize()
of the underlying buffers exceeds maximum size of Java array, anRuntimeException
will be thrown.
-
copyTo
public void copyTo(DataOutput output) throws java.io.IOException
Copy the current content of this object into anotherDataOutput
.- Throws:
java.io.IOException
-
size
public long size()
- Returns:
- The number of bytes written to this output so far.
-
toString
public java.lang.String toString()
- Overrides:
toString
in classjava.lang.Object
-
writeShort
public void writeShort(short v)
Description copied from class:DataOutput
Writes a short as two bytes.- Overrides:
writeShort
in classDataOutput
- See Also:
DataInput.readShort()
-
writeInt
public void writeInt(int v)
Description copied from class:DataOutput
Writes an int as four bytes.32-bit unsigned integer written as four bytes, high-order bytes first.
- Overrides:
writeInt
in classDataOutput
- See Also:
DataInput.readInt()
-
writeLong
public void writeLong(long v)
Description copied from class:DataOutput
Writes a long as eight bytes.64-bit unsigned integer written as eight bytes, high-order bytes first.
- Overrides:
writeLong
in classDataOutput
- See Also:
DataInput.readLong()
-
writeString
public void writeString(java.lang.String v)
Description copied from class:DataOutput
Writes a string.Writes strings as UTF-8 encoded bytes. First the length, in bytes, is written as a
VInt
, followed by the bytes.- Overrides:
writeString
in classDataOutput
- See Also:
DataInput.readString()
-
writeMapOfStrings
public void writeMapOfStrings(java.util.Map<java.lang.String,java.lang.String> map)
Description copied from class:DataOutput
Writes a String map.First the size is written as an
vInt
, followed by each key-value pair written as two consecutiveString
s.- Overrides:
writeMapOfStrings
in classDataOutput
- Parameters:
map
- Input map.
-
writeSetOfStrings
public void writeSetOfStrings(java.util.Set<java.lang.String> set)
Description copied from class:DataOutput
Writes a String set.First the size is written as an
vInt
, followed by each value written as aString
.- Overrides:
writeSetOfStrings
in classDataOutput
- Parameters:
set
- Input set.
-
ramBytesUsed
public long ramBytesUsed()
Description copied from interface:Accountable
Return the memory usage of this object in bytes. Negative values are illegal.- Specified by:
ramBytesUsed
in interfaceAccountable
-
reset
public void reset()
This method resets this object to a clean (zero-size) state and publishes any currently allocated buffers for reuse to the reuse strategy provided in the constructor. Sharing byte buffers for reads and writes is dangerous and will very likely lead to hard-to-debug issues, use with great care.
-
newResettableInstance
public static ByteBuffersDataOutput newResettableInstance()
- Returns:
- Returns a new
ByteBuffersDataOutput
with thereset()
capability.
-
blockSize
private int blockSize()
-
appendBlock
private void appendBlock()
-
rewriteToBlockSize
private void rewriteToBlockSize(int targetBlockBits)
-
computeBlockSizeBitsFor
private static int computeBlockSizeBitsFor(long bytes)
-
UTF16toUTF8
private static int UTF16toUTF8(java.lang.CharSequence s, int offset, int length, byte[] buf, java.util.function.IntConsumer bufferFlusher)
A consumer-based UTF16-UTF8 encoder (writes the input string in smaller buffers.).
-
-