Package org.apache.cassandra.service.paxos

Class PaxosRepair

  • public class PaxosRepair
extends AbstractPaxosRepair
    Facility to finish any in-progress paxos transaction, and ensure that a quorum of nodes agree on the most recent operation. Semantically, we simply ensure that any side effects that were "decided" before repair was initiated have been committed to a quorum of nodes. This means: - any prepare that has _possibly_ reached a quorum of nodes will be invalidated - any proposal that has been accepted by at least one node, but not known to be committed to any, will be proposed again - any proposal that has been committed to at least one node, but not committed to all, will be committed to a quorum Note that once started, this continues to try to repair any ongoing operations for the partition up to 4 times. In a functioning cluster this should always be possible, but during a network partition this might cause the repair to fail. Requirements for correction: - If performed during a range movement, we depend on a quorum (of the new topology) have been informed of the new topology _prior_ to initiating this repair, and this node to have been a member of a quorum of nodes verifying - If a quorum of nodes is unaware of the new topology prior to initiating repair, an operation could simply occur after repair completes that permits a linearization failure, such as with CASSANDRA-15745. their topology is up-to-date. - Paxos prepare rounds must also verify the topology being used with their peers - If prepare rounds do not verify their topology, a node that is not a member of the quorum who have agreed the latest topology could still perform an operation without being aware of the topology change, and permit a linearization failure, such as with CASSANDRA-15745. With these issues addressed elsewhere, our algorithm is fairly simple. In brief: 1) Query all replicas for any promises or proposals they have witnessed that have not been committed, and their most recent commit. Wait for a quorum of responses. 2) If this is the first time we have queried other nodes, we take a note of the most recent ballot we see; If this is not the first time we have queried other nodes, and we have committed a newer ballot than the one we previously recorded, we terminate (somebody else has done the work for us). 3) If we see an in-progress operation that is very recent, we wait for it to complete and try again 4) If we see a previously accepted operation, we attempt to complete it, or if we see a prepare with no proposal, we propose an empty update to invalidate it; otherwise we have nothing to do, as there is no operation that can have produced a side-effect before we began. 5) We prepare a paxos round to agree the new commit using a higher ballot than the one witnessed, but a lower than one we would propose a new operation with. This permits newer operations to "beat" us so that we do not interfere with normal paxos operations. 6) If we are "beaten" we start again (without delay, as (2) manages delays where necessary)

    • Method Detail

      • toString

        public java.lang.String toString()
        Overrides:
        toString in class java.lang.Object

      • hasSufficientLiveNodesForTopologyChange

        public static boolean hasSufficientLiveNodesForTopologyChange​(Keyspace keyspace,
                                                              Range<Token> range,
                                                              java.util.Collection<InetAddressAndPort> liveEndpoints)
        checks if we have enough live nodes to perform a paxos repair for topology repair. Generally, this means that we need enough live participants to reach EACH_QUORUM, with a few exceptions. The EACH_QUORUM requirement is meant to support workload using either SERIAL or LOCAL_SERIAL if paxos_topology_repair_strict_each_quorum is set to false (the default), we will accept either a quorum or n-1 live nodes in the cluster and per dc. If paxos_topology_repair_no_dc_checks is true, we only check the live nodes in the cluster, and do not do any per-dc checks.

      • setSkipPaxosRepairCompatibilityCheck

        public static void setSkipPaxosRepairCompatibilityCheck​(boolean v)

      • getSkipPaxosRepairCompatibilityCheck

        public static boolean getSkipPaxosRepairCompatibilityCheck()

      • validatePeerCompatibility

        public static boolean validatePeerCompatibility​(TableMetadata table,
                                                java.util.Collection<Range<Token>> ranges)

      • shutdownAndWait

        public static void shutdownAndWait​(long timeout,
                                   java.util.concurrent.TimeUnit units)
                            throws java.lang.InterruptedException,
                                   java.util.concurrent.TimeoutException
        Throws:
        java.lang.InterruptedException
        java.util.concurrent.TimeoutException