Class BinaryDictionaryWriter
- java.lang.Object
-
- org.apache.lucene.analysis.ja.util.BinaryDictionaryWriter
-
- Direct Known Subclasses:
TokenInfoDictionaryWriter
,UnknownDictionaryWriter
abstract class BinaryDictionaryWriter extends java.lang.Object
-
-
Field Summary
Fields Modifier and Type Field Description protected java.nio.ByteBuffer
buffer
private static int
ID_LIMIT
private java.lang.Class<? extends BinaryDictionary>
implClazz
private int
lastSourceId
private int
lastWordId
private java.util.ArrayList<java.lang.String>
posDict
private int[]
targetMap
private int
targetMapEndOffset
private int[]
targetMapOffsets
-
Constructor Summary
Constructors Constructor Description BinaryDictionaryWriter(java.lang.Class<? extends BinaryDictionary> implClazz, int size)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description (package private) void
addMapping(int sourceId, int wordId)
(package private) java.lang.String
getBaseFileName()
private boolean
isKatakana(java.lang.String s)
int
put(java.lang.String[] entry)
put the entry in mapprivate static int
sharedPrefix(java.lang.String left, java.lang.String right)
private java.lang.String
toKatakana(java.lang.String s)
void
write(java.nio.file.Path baseDir)
Write dictionary in file Dictionary format is: [Size of dictionary(int)], [entry:{left id(short)}{right id(short)}{word cost(short)}{length of pos info(short)}{pos info(char)}], [entry...], [entry...].....private void
writeDictionary(java.nio.file.Path path)
private void
writeKatakana(java.lang.String s)
private void
writePosDict(java.nio.file.Path path)
private void
writeTargetMap(java.nio.file.Path path)
-
-
-
Field Detail
-
ID_LIMIT
private static final int ID_LIMIT
- See Also:
- Constant Field Values
-
implClazz
private final java.lang.Class<? extends BinaryDictionary> implClazz
-
buffer
protected java.nio.ByteBuffer buffer
-
targetMapEndOffset
private int targetMapEndOffset
-
lastWordId
private int lastWordId
-
lastSourceId
private int lastSourceId
-
targetMap
private int[] targetMap
-
targetMapOffsets
private int[] targetMapOffsets
-
posDict
private final java.util.ArrayList<java.lang.String> posDict
-
-
Constructor Detail
-
BinaryDictionaryWriter
BinaryDictionaryWriter(java.lang.Class<? extends BinaryDictionary> implClazz, int size)
-
-
Method Detail
-
put
public int put(java.lang.String[] entry)
put the entry in map- Returns:
- current position of buffer, which will be wordId of next entry
-
isKatakana
private boolean isKatakana(java.lang.String s)
-
writeKatakana
private void writeKatakana(java.lang.String s)
-
toKatakana
private java.lang.String toKatakana(java.lang.String s)
-
sharedPrefix
private static int sharedPrefix(java.lang.String left, java.lang.String right)
-
addMapping
void addMapping(int sourceId, int wordId)
-
getBaseFileName
final java.lang.String getBaseFileName()
-
write
public void write(java.nio.file.Path baseDir) throws java.io.IOException
Write dictionary in file Dictionary format is: [Size of dictionary(int)], [entry:{left id(short)}{right id(short)}{word cost(short)}{length of pos info(short)}{pos info(char)}], [entry...], [entry...].....- Throws:
java.io.IOException
- if an I/O error occurs writing the dictionary files
-
writeTargetMap
private void writeTargetMap(java.nio.file.Path path) throws java.io.IOException
- Throws:
java.io.IOException
-
writePosDict
private void writePosDict(java.nio.file.Path path) throws java.io.IOException
- Throws:
java.io.IOException
-
writeDictionary
private void writeDictionary(java.nio.file.Path path) throws java.io.IOException
- Throws:
java.io.IOException
-
-