E2E: ensure ConnectACLsE2ETest has clean state before starting (#12334)

The `ConnectACLsE2ETest` checks that the SI tokens have been properly
cleaned up between tests, but following the change to use HCP the
previous `Connect` test suite will often have SI tokens that haven't
been cleaned up by the time this test suite runs. Wait for the SI
tokens to be cleaned up at the start of the test to ensure we have a
clean state.

e2e/connect/acls.go

@@ -40,6 +40,13 @@ func (tc *ConnectACLsE2ETest) BeforeAll(f *framework.F) {
 
 	_, err := uuidparse.ParseUUID(tc.consulManagementToken)
 	f.NoError(err, "CONSUL_HTTP_TOKEN not set")
+
+	// ensure SI tokens from previous test cases were removed
+	f.Eventually(func() bool {
+		siTokens := tc.countSITokens(f.T())
+		f.T().Log("cleanup: checking for remaining SI tokens:", siTokens)
+		return len(siTokens) == 0
+	}, 2*time.Minute, 2*time.Second, "SI tokens did not get removed")
 }
 
 // AfterEach does cleanup of Consul ACL objects that were created during each