Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SOLR-17015: Test and fix bug in MoveReplicaCmd for PRS collections #155

Open
wants to merge 1 commit into
base: fs/branch_9_3
Choose a base branch
from
Open

SOLR-17015: Test and fix bug in MoveReplicaCmd for PRS collections #155

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 0 additions & 7 deletions solr/core/src/java/org/apache/solr/cloud/ActiveReplicaWatcher.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,6 @@ public class ActiveReplicaWatcher implements CollectionStateWatcher {
private final List<String> solrCoreNames = new ArrayList<>();
private final List<Replica> activeReplicas = new ArrayList<>();

private int lastZkVersion = -1;

private SolrCloseableLatch latch;

/**
Expand Down Expand Up @@ -149,11 +147,6 @@ public synchronized boolean onStateChanged(Set<String> liveNodes, DocCollection
log.debug("-- already done, exiting...");
return true;
}
if (collectionState.getZNodeVersion() == lastZkVersion) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this check for non-prs collection?

Copy link
Collaborator

@patsonluk patsonluk Oct 6, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just want to confirm my understanding, this check did not work for PRS enabled collection because PRS entries change would not update the zNodeVersion, hence PRS updates are ignored and the watch can stall because it never sees the active replica?

If that's the case, i'm wondering if we can check both the node and the child version if collectionState.isPerReplicaState() is true?

I looked at the original change of this and it's a part of a pretty big PR so I don't know the importance of this check (like did we really see alot of spurious calls?). But erring on the safe side, perhaps we can keep it and make it work for PRS too :) ?

As for reference, I searched message that contains seconds for replica to become active (AddReplica timeout if this was an issue) on BQ solros nodes (sep and oct), so far we don't have any hits yet.

But regardless, this is definitely an issue for PRS enabled collections and need to be fixed! 😊

log.debug("-- spurious call with already seen zkVersion= {}, ignoring...", lastZkVersion);
return false;
}
lastZkVersion = collectionState.getZNodeVersion();

for (Slice slice : collectionState.getSlices()) {
for (Replica replica : slice.getReplicas()) {
Expand Down
3 changes: 1 addition & 2 deletions solr/core/src/test/org/apache/solr/cloud/MoveReplicaTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,9 @@
*/
public class MoveReplicaTest extends AbstractMoveReplicaTestBase {

@Override
@Test
public void test() throws Exception {
super.test();
super.test(random().nextBoolean());
}

@Override
Expand Down
3 changes: 1 addition & 2 deletions solr/modules/hdfs/src/test/org/apache/solr/hdfs/cloud/MoveReplicaHdfsTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,8 +65,7 @@ protected String getConfigSet() {

@Test
public void testNormalMove() throws Exception {
inPlaceMove = false;
super.test();
super.test(inPlaceMove = false);
}

@Test
Expand Down
6 changes: 4 additions & 2 deletions solr/solrj-zookeeper/src/java/org/apache/solr/common/cloud/ZkStateReader.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1806,9 +1806,11 @@ public void waitForState(
AtomicReference<DocCollection> docCollection = new AtomicReference<>();
CollectionStateWatcher watcher =
(n, c) -> {
docCollection.set(c);
boolean matches = predicate.matches(n, c);
if (matches) latch.countDown();
if (matches) {
docCollection.set(c);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may need to keep this outside? what if condition doens't match and timeout happens below. We are accessing collection below

throw new TimeoutException(
            "Timeout waiting to see state for collection="
                + collection
                + " :"
                + docCollection.get());

latch.countDown();
}

return matches;
};
Expand Down
Loading