Package org.apache.cassandra.db

Class Directories

  • public class Directories
extends java.lang.Object
    Encapsulate handling of paths to the data files. 
     
   /<path_to_data_dir>/ks/<cf dir>/ks-cf1-jb-1-Data.db
                         /<cf dir>/la-2-Data.db
                         /<cf dir>/.<index name>/ks-cf1.idx-jb-1-Data.db
                         /<cf dir>/.<index name>/la-1-Data.db
                         ...
    Until v2.0, <cf dir> is just column family name. Since v2.1, <cf dir> has column family ID(tableId) added to its end. SSTables from secondary indexes were put in the same directory as their parent. Since v2.2, they have their own directory under the parent directory whose name is index name. Upon startup, those secondary index files are moved to new directory when upgrading. For backward compatibility, Directories can use directory without tableId if exists. In addition, more that one 'root' data directory can be specified so that <path_to_data_dir> potentially represents multiple locations. Note that in the case of multiple locations, the manifest for the leveled compaction is only in one of the location. Snapshots (resp. backups) are always created along the sstables there are snapshotted (resp. backuped) but inside a subdirectory named 'snapshots' (resp. backups) (and snapshots are further inside a subdirectory of the name of the snapshot). For secondary indexes, snapshots (backups) are not created in their own directory, but are in their parent's snapshot (backup) directory. This class abstracts all those details from the rest of the code.

    • Field Detail

      • SECONDARY_INDEX_NAME_SEPARATOR

        public static final java.lang.String SECONDARY_INDEX_NAME_SEPARATOR
        See Also:
        Constant Field Values

      • TABLE_DIRECTORY_NAME_SEPARATOR

        public static final java.lang.String TABLE_DIRECTORY_NAME_SEPARATOR
        See Also:
        Constant Field Values

    • Constructor Detail

      • Directories

        public Directories​(TableMetadata metadata,
                   Directories.DataDirectory[] paths)
        Create Directories of given ColumnFamily. SSTable directories are created under data_directories defined in cassandra.yaml if not exist at this time.
        Parameters:
        metadata - metadata of ColumnFamily

    • Method Detail

      • verifyFullPermissions

        public static boolean verifyFullPermissions​(File dir,
                                            java.lang.String dataDir)
        Checks whether Cassandra has RWX permissions to the specified directory. Logs an error with the details if it does not.
        Parameters:
        dir - File object of the directory.
        dataDir - String representation of the directory's location
        Returns:
        status representing Cassandra's RWX permissions to the supplied folder location.

      • getLocationForDisk

        public File getLocationForDisk​(Directories.DataDirectory dataDirectory)
        Returns SSTable location which is inside given data directory.
        Parameters:
        dataDirectory -
        Returns:
        SSTable location

      • find

        public Descriptor find​(java.lang.String filename)

      • getDirectoryForNewSSTables

        public File getDirectoryForNewSSTables()
        Basically the same as calling getWriteableLocationAsFile(long) with an unknown size (-1L), which may return any allowed directory - even a data directory that has no usable space. Do not use this method in production code.
        Throws:
        FSWriteError - if all directories are disallowed.

      • getWriteableLocationAsFile

        public File getWriteableLocationAsFile​(long writeSize)
        Returns an allowed directory that _currently_ has writeSize bytes as usable space.
        Throws:
        FSWriteError - if all directories are disallowed.

      • getWriteableLocationToLoadFile

        public File getWriteableLocationToLoadFile​(File sourceFile)
        Returns a data directory to load the file sourceFile. If the sourceFile is on same disk partition as any data directory then use that one as data directory otherwise use getWriteableLocationAsFile(long) to find suitable data directory. Also makes sure returned directory is not disallowed.
        Throws:
        FSWriteError - if all directories are disallowed.

      • getTemporaryWriteableDirectoryAsFile

        public File getTemporaryWriteableDirectoryAsFile​(long writeSize)
        Returns a temporary subdirectory on allowed data directory that _currently_ has writeSize bytes as usable space. This method does not create the temporary directory.
        Throws:
        java.io.IOError - if all directories are disallowed.

      • removeTemporaryDirectories

        public void removeTemporaryDirectories()

      • getWriteableLocation

        public Directories.DataDirectory getWriteableLocation​(long writeSize)
        Returns an allowed data directory that _currently_ has writeSize bytes as usable space.
        Throws:
        FSWriteError - if all directories are disallowed.

      • hasAvailableDiskSpace

        public boolean hasAvailableDiskSpace​(long estimatedSSTables,
                                     long expectedTotalWriteSize)

      • getSnapshotDirectory

        public static File getSnapshotDirectory​(Descriptor desc,
                                        java.lang.String snapshotName)

      • getSnapshotDirectory

        public static File getSnapshotDirectory​(File location,
                                        java.lang.String snapshotName)
        Returns directory to write snapshot. If directory does not exist, then one is created. If given location indicates secondary index, this will return <cf dir>/snapshots/<snapshot name>/.<index name>. Otherwise, this will return <cf dir>/snapshots/<snapshot name>.
        Parameters:
        location - base directory
        snapshotName - snapshot name
        Returns:
        directory to write snapshot

      • getSnapshotManifestFile

        public File getSnapshotManifestFile​(java.lang.String snapshotName)

      • getSnapshotManifestFile

        public static File getSnapshotManifestFile​(File snapshotDir)

      • getSnapshotSchemaFile

        public File getSnapshotSchemaFile​(java.lang.String snapshotName)

      • getSnapshotSchemaFile

        public static File getSnapshotSchemaFile​(File snapshotDir)

      • getNewEphemeralSnapshotMarkerFile

        public File getNewEphemeralSnapshotMarkerFile​(java.lang.String snapshotName)

      • getBackupsDirectory

        public static File getBackupsDirectory​(Descriptor desc)

      • getBackupsDirectory

        public static File getBackupsDirectory​(File location)

      • isStoredInLocalSystemKeyspacesDataLocation

        public static boolean isStoredInLocalSystemKeyspacesDataLocation​(java.lang.String keyspace,
                                                                 java.lang.String table)
        Checks if the specified table should be stored with local system data.

        To minimize the risk of failures, SSTables for local system keyspaces must be stored in a single data directory. The only exception to this are some of the system table as the server can continue operating even if those tables loose some data.

        Parameters:
        keyspace - the keyspace name
        table - the table name
        Returns:
        true if the specified table should be stored with local system data, false otherwise.

      • listSnapshots

        public java.util.Map<java.lang.String,​TableSnapshot> listSnapshots()

      • maybeLoadManifest

        protected static SnapshotManifest maybeLoadManifest​(java.lang.String keyspace,
                                                    java.lang.String table,
                                                    java.lang.String tag,
                                                    java.util.Set<File> snapshotDirs)

      • listEphemeralSnapshots

        public java.util.List<java.lang.String> listEphemeralSnapshots()

      • listSnapshotDirsByTag

        protected java.util.Map<java.lang.String,​java.util.Set<File>> listSnapshotDirsByTag()

      • snapshotExists

        public boolean snapshotExists​(java.lang.String snapshotName)

      • clearSnapshot

        public static void clearSnapshot​(java.lang.String snapshotName,
                                 java.util.List<File> tableDirectories,
                                 com.google.common.util.concurrent.RateLimiter snapshotRateLimiter)

      • removeSnapshotDirectory

        public static void removeSnapshotDirectory​(com.google.common.util.concurrent.RateLimiter snapshotRateLimiter,
                                           File snapshotDir)

      • trueSnapshotsSize

        public long trueSnapshotsSize()
        Returns:
        total snapshot size in byte for all snapshots.

      • getRawDiretoriesSize

        public long getRawDiretoriesSize()
        Returns:
        Raw size on disk for all directories

      • getTrueAllocatedSizeIn

        public long getTrueAllocatedSizeIn​(File snapshotDir)

      • getKSChildDirectories

        public static java.util.List<File> getKSChildDirectories​(java.lang.String ksName)

      • isSecondaryIndexFolder

        public static boolean isSecondaryIndexFolder​(File dir)

      • isSecondaryIndexFolder

        public static boolean isSecondaryIndexFolder​(java.nio.file.Path dir)

      • getCFDirectories

        public java.util.List<File> getCFDirectories()

      • getUIDGenerator

        public <T extends SSTableId> java.util.function.Supplier<T> getUIDGenerator​(SSTableId.Builder<T> builder)
        Initializes the sstable unique identifier generator using a provided builder for this instance of directories. If the id builder needs that, sstables in these directories are listed to provide the existing identifiers to the builder. The listing is done lazily so if the builder does not require that, listing is skipped.