Package org.apache.lucene.index
Class DocumentsWriterPerThread
- java.lang.Object
-
- org.apache.lucene.index.DocumentsWriterPerThread
-
final class DocumentsWriterPerThread extends java.lang.Object
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description (package private) static class
DocumentsWriterPerThread.FlushedSegment
(package private) static class
DocumentsWriterPerThread.IndexingChain
The IndexingChain must define theDocumentsWriterPerThread.IndexingChain.getChain(DocumentsWriterPerThread)
method which returns the DocConsumer that the DocumentsWriter calls to process the documents.private static class
DocumentsWriterPerThread.IntBlockAllocator
-
Field Summary
Fields Modifier and Type Field Description private boolean
aborted
private java.lang.Throwable
abortingException
(package private) static int
BYTE_BLOCK_NOT_MASK
(package private) ByteBlockPool.Allocator
byteBlockAllocator
(package private) Counter
bytesUsed
(package private) Codec
codec
private DocConsumer
consumer
(package private) static DocumentsWriterPerThread.IndexingChain
defaultIndexingChain
private int[]
deleteDocIDs
(package private) DocumentsWriterDeleteQueue
deleteQueue
private DocumentsWriterDeleteQueue.DeleteSlice
deleteSlice
(package private) TrackingDirectoryWrapper
directory
private boolean
enableTestPoints
private FieldInfos.Builder
fieldInfos
private java.util.Set<java.lang.String>
filesToDelete
private SetOnce<java.lang.Boolean>
flushPending
private SetOnce<java.lang.Boolean>
hasFlushed
private int
indexVersionCreated
private LiveIndexWriterConfig
indexWriterConfig
private static boolean
INFO_VERBOSE
private InfoStream
infoStream
(package private) IntBlockPool.Allocator
intBlockAllocator
private long
lastCommittedBytesUsed
private java.util.concurrent.locks.ReentrantLock
lock
(package private) static int
MAX_TERM_LENGTH_UTF8
private java.text.NumberFormat
nf
private int
numDeletedDocIds
private int
numDocsInRAM
private java.util.concurrent.atomic.AtomicLong
pendingNumDocs
private BufferedUpdates
pendingUpdates
private SegmentInfo
segmentInfo
-
Constructor Summary
Constructors Constructor Description DocumentsWriterPerThread(int indexVersionCreated, java.lang.String segmentName, Directory directoryOrig, Directory directory, LiveIndexWriterConfig indexWriterConfig, DocumentsWriterDeleteQueue deleteQueue, FieldInfos.Builder fieldInfos, java.util.concurrent.atomic.AtomicLong pendingNumDocs, boolean enableTestPoints)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description (package private) void
abort()
Called if we hit an exception at a bad time (when updating the index files) and must discard all currently buffered docs.(package private) long
bytesUsed()
(package private) long
commitLastBytesUsed()
Commits the currentbytesUsed()
and stores it's value for later reuse.private void
deleteLastDocs(int docCount)
private long
finishDocuments(DocumentsWriterDeleteQueue.Node<?> deleteNode, int docIdUpTo)
(package private) DocumentsWriterPerThread.FlushedSegment
flush(DocumentsWriter.FlushNotifications flushNotifications)
Flush all pending docs to a new segment(package private) FieldInfos.Builder
getFieldInfosBuilder()
(package private) int
getIndexCreatedVersionMajor()
(package private) LiveIndexWriterConfig
getIndexWriterConfig()
(package private) long
getLastCommittedBytesUsed()
Returns the last committed bytes for this DWPT.int
getNumDocsInRAM()
Returns the number of RAM resident documents in thisDocumentsWriterPerThread
(package private) SegmentInfo
getSegmentInfo()
Get current segment info we are writing.(package private) boolean
hasFlushed()
Returnstrue
iff this DWPT has been flushed(package private) boolean
hasHitAbortingException()
(package private) boolean
isAborted()
(package private) boolean
isFlushPending()
Returns true iff this DWPT is marked as flush pending(package private) boolean
isHeldByCurrentThread()
Returns true if the DWPT's lock is held by the current thread(package private) void
lock()
Locks this DWPT for exclusive access.private void
maybeAbort(java.lang.String location, DocumentsWriter.FlushNotifications flushNotifications)
(package private) void
onAbortingException(java.lang.Throwable throwable)
(package private) java.util.Set<java.lang.String>
pendingFilesToDelete()
(package private) FrozenBufferedUpdates
prepareFlush()
Prepares this DWPT for flushing.private void
reserveOneDoc()
Anything that will add N docs to the index should reserve first to make sure it's allowed.(package private) void
sealFlushedSegment(DocumentsWriterPerThread.FlushedSegment flushedSegment, Sorter.DocMap sortMap, DocumentsWriter.FlushNotifications flushNotifications)
Seals theSegmentInfo
for the new flushed segment and persists the deleted documentsFixedBitSet
.(package private) void
setFlushPending()
Sets this DWPT as flush pending.private FixedBitSet
sortLiveDocs(Bits liveDocs, Sorter.DocMap sortMap)
(package private) void
testPoint(java.lang.String message)
java.lang.String
toString()
(package private) boolean
tryLock()
Acquires the DWPT's lock only if it is not held by another thread at the time of invocation.(package private) void
unlock()
Unlocks the DWPT's lock(package private) long
updateDocuments(java.lang.Iterable<? extends java.lang.Iterable<? extends IndexableField>> docs, DocumentsWriterDeleteQueue.Node<?> deleteNode, DocumentsWriter.FlushNotifications flushNotifications)
-
-
-
Field Detail
-
abortingException
private java.lang.Throwable abortingException
-
defaultIndexingChain
static final DocumentsWriterPerThread.IndexingChain defaultIndexingChain
-
INFO_VERBOSE
private static final boolean INFO_VERBOSE
- See Also:
- Constant Field Values
-
codec
final Codec codec
-
directory
final TrackingDirectoryWrapper directory
-
consumer
private final DocConsumer consumer
-
bytesUsed
final Counter bytesUsed
-
pendingUpdates
private final BufferedUpdates pendingUpdates
-
segmentInfo
private final SegmentInfo segmentInfo
-
aborted
private boolean aborted
-
flushPending
private SetOnce<java.lang.Boolean> flushPending
-
lastCommittedBytesUsed
private volatile long lastCommittedBytesUsed
-
hasFlushed
private SetOnce<java.lang.Boolean> hasFlushed
-
fieldInfos
private final FieldInfos.Builder fieldInfos
-
infoStream
private final InfoStream infoStream
-
numDocsInRAM
private int numDocsInRAM
-
deleteQueue
final DocumentsWriterDeleteQueue deleteQueue
-
deleteSlice
private final DocumentsWriterDeleteQueue.DeleteSlice deleteSlice
-
nf
private final java.text.NumberFormat nf
-
byteBlockAllocator
final ByteBlockPool.Allocator byteBlockAllocator
-
intBlockAllocator
final IntBlockPool.Allocator intBlockAllocator
-
pendingNumDocs
private final java.util.concurrent.atomic.AtomicLong pendingNumDocs
-
indexWriterConfig
private final LiveIndexWriterConfig indexWriterConfig
-
enableTestPoints
private final boolean enableTestPoints
-
indexVersionCreated
private final int indexVersionCreated
-
lock
private final java.util.concurrent.locks.ReentrantLock lock
-
deleteDocIDs
private int[] deleteDocIDs
-
numDeletedDocIds
private int numDeletedDocIds
-
filesToDelete
private final java.util.Set<java.lang.String> filesToDelete
-
BYTE_BLOCK_NOT_MASK
static final int BYTE_BLOCK_NOT_MASK
- See Also:
- Constant Field Values
-
MAX_TERM_LENGTH_UTF8
static final int MAX_TERM_LENGTH_UTF8
- See Also:
- Constant Field Values
-
-
Constructor Detail
-
DocumentsWriterPerThread
DocumentsWriterPerThread(int indexVersionCreated, java.lang.String segmentName, Directory directoryOrig, Directory directory, LiveIndexWriterConfig indexWriterConfig, DocumentsWriterDeleteQueue deleteQueue, FieldInfos.Builder fieldInfos, java.util.concurrent.atomic.AtomicLong pendingNumDocs, boolean enableTestPoints) throws java.io.IOException
- Throws:
java.io.IOException
-
-
Method Detail
-
getIndexWriterConfig
LiveIndexWriterConfig getIndexWriterConfig()
-
onAbortingException
final void onAbortingException(java.lang.Throwable throwable)
-
hasHitAbortingException
final boolean hasHitAbortingException()
-
isAborted
final boolean isAborted()
-
abort
void abort() throws java.io.IOException
Called if we hit an exception at a bad time (when updating the index files) and must discard all currently buffered docs. This resets our state, discarding any docs added since last flush.- Throws:
java.io.IOException
-
getFieldInfosBuilder
FieldInfos.Builder getFieldInfosBuilder()
-
getIndexCreatedVersionMajor
int getIndexCreatedVersionMajor()
-
testPoint
final void testPoint(java.lang.String message)
-
reserveOneDoc
private void reserveOneDoc()
Anything that will add N docs to the index should reserve first to make sure it's allowed.
-
updateDocuments
long updateDocuments(java.lang.Iterable<? extends java.lang.Iterable<? extends IndexableField>> docs, DocumentsWriterDeleteQueue.Node<?> deleteNode, DocumentsWriter.FlushNotifications flushNotifications) throws java.io.IOException
- Throws:
java.io.IOException
-
finishDocuments
private long finishDocuments(DocumentsWriterDeleteQueue.Node<?> deleteNode, int docIdUpTo)
-
deleteLastDocs
private void deleteLastDocs(int docCount)
-
getNumDocsInRAM
public int getNumDocsInRAM()
Returns the number of RAM resident documents in thisDocumentsWriterPerThread
-
prepareFlush
FrozenBufferedUpdates prepareFlush()
Prepares this DWPT for flushing. This method will freeze and return theDocumentsWriterDeleteQueue
s global buffer and apply all pending deletes to this DWPT.
-
flush
DocumentsWriterPerThread.FlushedSegment flush(DocumentsWriter.FlushNotifications flushNotifications) throws java.io.IOException
Flush all pending docs to a new segment- Throws:
java.io.IOException
-
maybeAbort
private void maybeAbort(java.lang.String location, DocumentsWriter.FlushNotifications flushNotifications) throws java.io.IOException
- Throws:
java.io.IOException
-
pendingFilesToDelete
java.util.Set<java.lang.String> pendingFilesToDelete()
-
sortLiveDocs
private FixedBitSet sortLiveDocs(Bits liveDocs, Sorter.DocMap sortMap)
-
sealFlushedSegment
void sealFlushedSegment(DocumentsWriterPerThread.FlushedSegment flushedSegment, Sorter.DocMap sortMap, DocumentsWriter.FlushNotifications flushNotifications) throws java.io.IOException
Seals theSegmentInfo
for the new flushed segment and persists the deleted documentsFixedBitSet
.- Throws:
java.io.IOException
-
getSegmentInfo
SegmentInfo getSegmentInfo()
Get current segment info we are writing.
-
bytesUsed
long bytesUsed()
-
toString
public java.lang.String toString()
- Overrides:
toString
in classjava.lang.Object
-
isFlushPending
boolean isFlushPending()
Returns true iff this DWPT is marked as flush pending
-
setFlushPending
void setFlushPending()
Sets this DWPT as flush pending. This can only be set once.
-
getLastCommittedBytesUsed
long getLastCommittedBytesUsed()
Returns the last committed bytes for this DWPT. This method can be called without acquiring the DWPTs lock.
-
commitLastBytesUsed
long commitLastBytesUsed()
Commits the currentbytesUsed()
and stores it's value for later reuse. The last committed bytes used can be retrieved viagetLastCommittedBytesUsed()
- Returns:
- the delta between the current
bytesUsed()
and the currentgetLastCommittedBytesUsed()
-
lock
void lock()
Locks this DWPT for exclusive access.- See Also:
ReentrantLock.lock()
-
tryLock
boolean tryLock()
Acquires the DWPT's lock only if it is not held by another thread at the time of invocation.- Returns:
- true if the lock was acquired.
- See Also:
ReentrantLock.tryLock()
-
isHeldByCurrentThread
boolean isHeldByCurrentThread()
Returns true if the DWPT's lock is held by the current thread- See Also:
ReentrantLock.isHeldByCurrentThread()
-
unlock
void unlock()
Unlocks the DWPT's lock- See Also:
ReentrantLock.unlock()
-
hasFlushed
boolean hasFlushed()
Returnstrue
iff this DWPT has been flushed
-
-