Overview | Package | Class | Use | Tree | Deprecated | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## org.apache.lucene.store
Class FSDirectory

    java.lang.Object
      org.apache.lucene.store.Directory
          org.apache.lucene.store.FSDirectory

All Implemented Interfaces:
  Closeable

Direct Known Subclasses:
  MMapDirectory, NIOFSDirectory, SimpleFSDirectory

    public abstract class FSDirectory

extends Directory

Base class for Directory implementations that store index files in the file system. There are currently three core subclasses:

- `SimpleFSDirectory` is a straightforward implementation using java.io.RandomAccessFile. However, it has poor concurrent performance (multiple threads will bottleneck) as it synchronizes when multiple threads read from the same file.
- `NIOFSDirectory` uses java.nio's FileChannel's positional io when reading to avoid synchronization when reading from the same file. Unfortunately, due to a Windows-only Sun JRE bug this is a poor choice for Windows, but on all other platforms this is the preferred choice. Applications using `Thread.interrupt()` or `Future.cancel(boolean)` should use `SimpleFSDirectory` instead. See `NIOFSDirectory` java doc for details.
- `MMapDirectory` uses memory-mapped IO when reading. This is a good choice if you have plenty of virtual memory relative to your index size, eg if you are running on a 64 bit JRE, or you are running on a 32 bit JRE but your index sizes are small enough to fit into the virtual memory space. Java has currently the limitation of not being able to unmap files from user code. The files are unmapped, when GC releases the byte buffers. Due to this bug in Sun's JRE, MMapDirectory's `FSDirectory.FSIndexInput.close()` is unable to close the underlying OS file handle. Only when GC finally collects the underlying objects, which could be quite some time later, will the file handle be closed. This will consume additional transient disk usage: on Windows, attempts to delete or overwrite the files will result in an exception; on other platforms, which typically have a "delete on last close" semantics, while such operations will succeed, the bytes are still consuming space on disk. For many applications this limitation is not a problem (e.g. if you have plenty of disk space, and you don't rely on overwriting files on Windows) but it's still an important limitation to be aware of. This class supplies a (possibly dangerous) workaround mentioned in the bug report, which may fail on non-Sun JVMs. Applications using `Thread.interrupt()` or `Future.cancel(boolean)` should use `SimpleFSDirectory` instead. See `MMapDirectory` java doc for details.

Unfortunately, because of system peculiarities, there is no single overall best implementation. Therefore, we've added the `open(java.io.File)` method, to allow Lucene to choose the best FSDirectory implementation given your environment, and the known limitations of each implementation. For users who have no reason to prefer a specific implementation, it's best to simply use `open(java.io.File)`. For all others, you should instantiate the desired implementation directly.

The locking implementation is by default `NativeFSLockFactory`, but can be changed by passing in a custom `LockFactory` instance.

See Also:
  `Directory`

Nested Class Summary
`protected static class` | `FSDirectory.FSIndexInput` Base class for reading input from a RandomAccessFile
`protected static class` | `FSDirectory.FSIndexOutput` Writes output with `RandomAccessFile.write(byte[], int, int)`

Nested classes/interfaces inherited from class org.apache.lucene.store.Directory
`Directory.IndexInputSlicer`

Field Summary
`static int` | `DEFAULT_READ_CHUNK_SIZE` Default read chunk size.
`protected File` | `directory`
`protected Set<String>` | `staleFiles`

Fields inherited from class org.apache.lucene.store.Directory
`isOpen, lockFactory`

Constructor Summary
`protected` | `FSDirectory(File path, LockFactory lockFactory)` Create a new FSDirectory for the named location (ctor for subclasses).

Method Summary
`void` | `close()` Closes the store to future operations.
`IndexOutput` | `createOutput(String name, IOContext context)` Creates an IndexOutput for the file with the given name.
`void` | `deleteFile(String name)` Removes an existing file in the directory.
`protected void` | `ensureCanWrite(String name)`
`boolean` | `fileExists(String name)` Returns true iff a file with the given name exists.
`long` | `fileLength(String name)` Returns the length in bytes of a file in the directory.
`static long` | `fileModified(File directory, String name)` Returns the time the named file was last modified.
`protected void` | `fsync(String name)`
`File` | `getDirectory()`
`String` | `getLockID()` Return a string identifier that uniquely differentiates this Directory instance from other Directory instances.
`int` | `getReadChunkSize()` The maximum number of bytes to read at once from the underlying file during `BufferedIndexInput.readBytes(byte[], int, int)`.
`String[]` | `listAll()` Lists all files (not subdirectories) in the directory.
`static String[]` | `listAll(File dir)` Lists all files (not subdirectories) in the directory.
`protected void` | `onIndexOutputClosed(FSDirectory.FSIndexOutput io)`
`static FSDirectory` | `open(File path)` Creates an FSDirectory instance, trying to pick the best implementation given the current environment.
`static FSDirectory` | `open(File path, LockFactory lockFactory)` Just like `open(File)`, but allows you to also specify a custom `LockFactory`.
`void` | `setLockFactory(LockFactory lockFactory)` Set the LockFactory that this Directory instance should use for its locking implementation.
`void` | `setReadChunkSize(int chunkSize)` Sets the maximum number of bytes read at once from the underlying file during `BufferedIndexInput.readBytes(byte[], int, int)`.
`void` | `sync(Collection<String> names)` Ensure that any writes to these files are moved to stable storage.
`String` | `toString()` For debug output.

Methods inherited from class org.apache.lucene.store.Directory
`clearLock, copy, createSlicer, ensureOpen, getLockFactory, makeLock, openInput`

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

Field Detail

### DEFAULT_READ_CHUNK_SIZE

    public static final int DEFAULT_READ_CHUNK_SIZE

  Default read chunk size. This is a conditional default: on 32bit JVMs, it defaults to 100 MB. On 64bit JVMs, it's `Integer.MAX_VALUE`.

  See Also:
    `setReadChunkSize(int)`

### directory

    protected final File directory

### staleFiles

    protected final Set<String> staleFiles

Constructor Detail

### FSDirectory

    protected FSDirectory(File path,
                          LockFactory lockFactory)
                   throws IOException

  Create a new FSDirectory for the named location (ctor for subclasses).

  Parameters:
    `path` - the path of the directory
    `lockFactory` - the lock factory to use, or null for the default (`NativeFSLockFactory`);
  Throws:
    `IOException` - if there is a low-level I/O error

Method Detail

### open

    public static FSDirectory open(File path)
                            throws IOException

  Creates an FSDirectory instance, trying to pick the best implementation given the current environment. The directory returned uses the `NativeFSLockFactory`.

  Currently this returns `MMapDirectory` for most Solaris and Windows 64-bit JREs, `NIOFSDirectory` for other non-Windows JREs, and `SimpleFSDirectory` for other JREs on Windows. It is highly recommended that you consult the implementation's documentation for your platform before using this method.

  NOTE: this method may suddenly change which implementation is returned from release to release, in the event that higher performance defaults become possible; if the precise implementation is important to your application, please instantiate it directly, instead. For optimal performance you should consider using `MMapDirectory` on 64 bit JVMs.

  See above

  Throws:
    `IOException`

### open

    public static FSDirectory open(File path,
                                   LockFactory lockFactory)
                            throws IOException

  Just like `open(File)`, but allows you to also specify a custom `LockFactory`.

  Throws:
    `IOException`

### setLockFactory

    public void setLockFactory(LockFactory lockFactory)
                        throws IOException

  Description copied from class: `Directory`
  Set the LockFactory that this Directory instance should use for its locking implementation. Each * instance of LockFactory should only be used for one directory (ie, do not share a single instance across multiple Directories).

  Overrides:
    `setLockFactory` in class `Directory`
  Parameters:
    `lockFactory` - instance of `LockFactory`.
  Throws:
    `IOException`

### listAll

    public static String[] listAll(File dir)
                            throws IOException

  Lists all files (not subdirectories) in the directory. This method never returns null (throws `IOException` instead).

  Throws:
    `NoSuchDirectoryException` - if the directory does not exist, or does exist but is not a directory.
    `IOException` - if list() returns null

### listAll

    public String[] listAll()
                     throws IOException

  Lists all files (not subdirectories) in the directory.

  Specified by:
    `listAll` in class `Directory`
  Throws:
    `NoSuchDirectoryException` - if the directory is not prepared for any write operations (such as `Directory.createOutput(String, IOContext)`).
    `IOException` - in case of other IO errors
  See Also:
    `listAll(File)`

### fileExists

    public boolean fileExists(String name)

  Returns true iff a file with the given name exists.

  Specified by:
    `fileExists` in class `Directory`

### fileModified

    public static long fileModified(File directory,
                                    String name)

  Returns the time the named file was last modified.

### fileLength

    public long fileLength(String name)
                    throws IOException

  Returns the length in bytes of a file in the directory.

  Specified by:
    `fileLength` in class `Directory`
  Parameters:
    `name` - the name of the file for which to return the length.
  Throws:
    `FileNotFoundException` - if the file does not exist.
    `IOException` - if there was an IO error while retrieving the file's length.

### deleteFile

    public void deleteFile(String name)
                    throws IOException

  Removes an existing file in the directory.

  Specified by:
    `deleteFile` in class `Directory`
  Throws:
    `IOException`

### createOutput

    public IndexOutput createOutput(String name,
                                    IOContext context)
                             throws IOException

  Creates an IndexOutput for the file with the given name.

  Specified by:
    `createOutput` in class `Directory`
  Throws:
    `IOException`

### ensureCanWrite

    protected void ensureCanWrite(String name)
                           throws IOException

  Throws:
    `IOException`

### onIndexOutputClosed

    protected void onIndexOutputClosed(FSDirectory.FSIndexOutput io)

### sync

    public void sync(Collection<String> names)
              throws IOException

  Description copied from class: `Directory`
  Ensure that any writes to these files are moved to stable storage. Lucene uses this to properly commit changes to the index, to prevent a machine/OS crash from corrupting the index.
  NOTE: Clients may call this method for same files over and over again, so some impls might optimize for that. For other impls the operation can be a noop, for various reasons.

  Specified by:
    `sync` in class `Directory`
  Throws:
    `IOException`

### getLockID

    public String getLockID()

  Description copied from class: `Directory`
  Return a string identifier that uniquely differentiates this Directory instance from other Directory instances. This ID should be the same if two Directory instances (even in different JVMs and/or on different machines) are considered "the same index". This is how locking "scopes" to the right index.

  Overrides:
    `getLockID` in class `Directory`

### close

    public void close()

  Closes the store to future operations.

  Specified by:
    `close` in interface `Closeable`
  Specified by:
    `close` in class `Directory`

### getDirectory

    public File getDirectory()

  Returns:
    the underlying filesystem directory

### toString

    public String toString()

  For debug output.

  Overrides:
    `toString` in class `Directory`

### setReadChunkSize

    public final void setReadChunkSize(int chunkSize)

  Sets the maximum number of bytes read at once from the underlying file during `BufferedIndexInput.readBytes(byte[], int, int)`. The default value is `DEFAULT_READ_CHUNK_SIZE`;

  This was introduced due to Sun JVM Bug 6478546, which throws an incorrect OutOfMemoryError when attempting to read too many bytes at once. It only happens on 32bit JVMs with a large maximum heap size.

  Changes to this value will not impact any already-opened `FSDirectory.FSIndexInput`s. You should call this before attempting to open an index on the directory.

  NOTE: This value should be as large as possible to reduce any possible performance impact. If you still encounter an incorrect OutOfMemoryError, trying lowering the chunk size.

### getReadChunkSize

    public final int getReadChunkSize()

  The maximum number of bytes to read at once from the underlying file during `BufferedIndexInput.readBytes(byte[], int, int)`.

  See Also:
    `setReadChunkSize(int)`

### fsync

    protected void fsync(String name)
                  throws IOException

  Throws:
    `IOException`

Overview | Package | Class | Use | Tree | Deprecated | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

Copyright © 2000-2013 Apache Software Foundation. All Rights Reserved.