JGIT validate if repository is valid

Question

Is there a method to kill a clone operation mid-stream? I will just use the cloning to validate the repository? Is there any other way to test if the remote url/repository is valid?

Answer 1

you can invoke 'git ls-remote ' using JGIT. see at here

Sample code as below:

    final LsRemoteCommand lsCmd = new LsRemoteCommand(null);
    final List<String> repos = Arrays.asList(
            "https://github.com/MuchContact/java.git",
            "git@github.com:MuchContact/java.git");
    for (String gitRepo: repos){
        lsCmd.setRemote(gitRepo);
        System.out.println(lsCmd.call().toString());
    }

Answer 2

I'm using the following heuristic (to be improved further):

private final static String INFO_REFS_PATH = "info/refs";

public static boolean isValidRepository(URIish repoUri) {
  if (repoUri.isRemote()) {
    return isValidRemoteRepository(repoUri);
  } else {
    return isValidLocalRepository(repoUri);
  }
}

private static boolean isValidLocalRepository(URIish repoUri) {
  boolean result;
  try {
    result = new FileRepository(repoUri.getPath()).getObjectDatabase().exists();
  } catch (IOException e) {
    result = false;
  }
  return result;
}

private static boolean isValidRemoteRepository(URIish repoUri) {
  boolean result;

  if (repoUri.getScheme().toLowerCase().startsWith("http") ) {
    String path = repoUri.getPath();
    String newPath = path.endsWith("/")? path + INFO_REFS_PATH : path + "/" + INFO_REFS_PATH;
    URIish checkUri = repoUri.setPath(newPath);

    InputStream ins = null;
    try {
      URLConnection conn = new URL(checkUri.toString()).openConnection();
      conn.setReadTimeout(NETWORK_TIMEOUT_MSEC);
      ins = conn.getInputStream();
      result = true;
    } catch (Exception e) {
      result = false;
    } finally {
      try { ins.close(); } catch (Exception e) { /* ignore */ }
    }

  } else if (repoUri.getScheme().toLowerCase().startsWith("ssh") ) {

    RemoteSession ssh = null;
    Process exec = null;

    try {
      ssh = SshSessionFactory.getInstance().getSession(repoUri, null, FS.detect(), 5000);
      exec = ssh.exec("cd " + repoUri.getPath() +"; git rev-parse --git-dir", 5000);

      Integer exitValue = null;
      do {
        try {
          exitValue = exec.exitValue();
        } catch (Exception e) { 
          try{Thread.sleep(1000);}catch(Exception ee){}
        }
      } while (exitValue == null);

      result = exitValue == 0;

    } catch (Exception e) {
      result = false;

    } finally {
      try { exec.destroy(); } catch (Exception e) { /* ignore */ }
      try { ssh.disconnect(); } catch (Exception e) { /* ignore */ }
    }

  } else {
    // TODO need to implement tests for other schemas
    result = true;
  }
  return result;
}

This works well with bare and non-bare repositories.

Please note that there seems to be a problem with the URIish.isRemote() method. When you create an URIish from a file-URL, the host is not null but an empty string! URIish.isRemote() however returns true, if the host field is not null...

EDIT: Added ssh support to the isValidRemoteRepository() method.

Answer 3

I gave a look at the JGit sources and there does not seem to be present a method to check for the validity of the remote repo.

This is the call method of CloneCommand:

public Git call() throws JGitInternalException {
    try {
        URIish u = new URIish(uri);
        Repository repository = init(u);
        FetchResult result = fetch(repository, u);
        if (!noCheckout)
            checkout(repository, result);
        return new Git(repository);
    } catch (IOException ioe) {
        throw new JGitInternalException(ioe.getMessage(), ioe);
    } catch (InvalidRemoteException e) {
        throw new JGitInternalException(e.getMessage(), e);
    } catch (URISyntaxException e) {
        throw new JGitInternalException(e.getMessage(), e);
    }
}

In order to get if the remote URL is invalid, when catching a JGitInternalException e you may get the actual cause with e.getCause() to look for InvalidRemoteException or even URISyntaxException, but as you pointed out you end up cloning if it is actually valid; the library does not allow you to interrupt the operation.

Delving more deeply into the JGit code, the TransportLocal class has the open(URIsh,Repository,String) method you could have used to check if an InvalidRemoteException is thrown, but its constructor is not public. A do-it-yourself solution is required, alas. You could maybe start from the contents of the TransportLocal.open method I mentioned.

Answer 4

AFAIK JGit doesn't have an implementation of git fsck yet.

Answer 5

To anyone looking, I'm using the following, more generic approach, to validate a remote repository (the code is in C#, but it shouldn't be hard to convert it to java).

public static bool IsValidRemoteRepository(URIish repoUri, CredentialsProvider credentialsProvider = null)
{
    var repoPath = Path.Combine(Path.GetTempPath(), Path.GetFileNameWithoutExtension(Path.GetRandomFileName()));

    Directory.CreateDirectory(repoPath);

    var git = Git.Init().SetBare(true).SetDirectory(repoPath).Call();

    var config = git.GetRepository().GetConfig();
    config.SetString("remote", "origin", "url", repoUri.ToString());
    config.Save();

    try
    {
        var cmd = git.LsRemote();

        if (credentialsProvider != null)
        {
            cmd.SetCredentialsProvider(credentialsProvider);
        }

        cmd.SetRemote("origin").Call();
    }
    catch (TransportException e)
    {
        LastException = e;
        return false;
    }

    return true;
}

Answer 6

JGit is working on an implementation of the git fsck command, but this has not been released on mvnrepository.com as far as I can see.

Example on how it will look like, check this test case.

@Test
public void testHealthyRepo() throws Exception {
    RevCommit commit0 = git.commit().message("0").create();
    RevCommit commit1 = git.commit().message("1").parent(commit0).create();
    git.update("master", commit1);

    DfsFsck fsck = new DfsFsck(repo);
    FsckError errors = fsck.check(null);

    assertEquals(errors.getCorruptObjects().size(), 0);
    assertEquals(errors.getMissingObjects().size(), 0);
    assertEquals(errors.getCorruptIndices().size(), 0);
}

Answer 7

API for validating remote repositories

public boolean validateRepository(String repositoryURL, String username, String password) throws Exception {
    boolean result = false;
    Repository db = FileRepositoryBuilder.create(new File("/tmp"));
    Git git = Git.wrap(db);
    final LsRemoteCommand lsCmd = git.lsRemote();
    lsCmd.setRemote(repositoryURL);
    if (username != null && password != null) {
        lsCmd.setCredentialsProvider(new UsernamePasswordCredentialsProvider(username, password));
    }
    if (null != lsCmd.call()){
        result = true;
    }
    return result;
}

Note : ls-remote api of jgit throws NPE which is known bug. So added the workaround as mentioned in the comments of the bug.
https://bugs.eclipse.org/bugs/show_bug.cgi?id=436695