14

Our application needs a full list of the user's files and folders. We use files.list() via the Javascript library (essentially the same code as shown in the official API reference as an example).

We use the "drive.files" scope.

Examining the response to the list, we find that some files are always missing. I did various tests to understand the problem:

  • The files clearly exist. They show up in the Google Drive Webapp and, if I explicitly request them via ID, I can get them via the API without problems.
  • It's reproducible, always the same files are missing.
  • It is not transient. I tried a day after and still the same files are missing. I know of a few strange effects in the API that go away after some time but not this one.
  • It is not a one time thing (e.g. some weird things went wrong during upload). If I repeat with a completely different Google Account again files are missing. Of a small set of 147 uploaded files in one test 4 are missed by the files.list call, in another test with the same 147 files on another account 23 files are missing.
  • It only occurs when I use the drive.files scope. If I relax the scope to drive all files are returned. If look at "Details" in the Google Drive Webapp also the missing files are shown as created by our Application. So it does not seem that they lost their origin somehow.
  • It also occurs when I specify a search query. If I call files.list with a search term "q: modifiedDate > '2012-06-04T12:00:00'" which also should return all files, the same files are missing.
  • I re-implemented the same thing as pure REST call to the API to rule out that it is an issue with the Javascript library. The error remains.

Update: I could track it down to an issue with the paging and the maxResults parameter. If I use different values the API returns different number of items:

With maxResults=100 I get 100+100+7=207.

With maxResults=99 I get 99+99+28=226.

With maxResults=101 I get 101+101+0=202.

The last result is interesting which gave me a nextLink indicating there are more results but the items array in the last response was actually empty. This alone might indicate a bug.

Still, this only occurs in drive.file scope, the counts are consistent in the full drive scope.

I'd be glad to hear ideas for a workaround. I'm aware of other ways to keep track of the users files, e.g. using the changes feed. I'm using that already but for a specific part in our application I simply need a reliable and complete list of all our application's items in a user's account.

One more note: We had other issues with the "drive.files" scope before (see Listing files with search query returns out-of-scope results (drive.files.list call, using drive.files scope)). This turned out to be an easy fix. Perhaps this issue is related.

Community
  • 1
  • 1
Lenau
  • 191
  • 4
  • Sounds like a bug, we're looking at it. – Burcu Dogan Apr 25 '13 at 22:19
  • @BurcuDogan Is there any update on that? Is there a changes page I can check from time to time to see if that's fixed? – Amir Uval Jun 11 '13 at 07:05
  • Are there any updates on this issue? We run repeatedly into this issue of unreliable file lists while we extend our Google Drive integration of our application. – Lenau Jul 17 '13 at 11:10
  • FYI, empty pages with a next link are normal, and you should use the presence of the next link, not the result size, as an indicator that there are more results to fetch. – Steve Bazyl Aug 06 '13 at 23:47
  • @SteveBazyl I understand that and that's exactly what I'm doing. It was just a side-note that it is strange that there is a next-link but when I actually get this page it is empty. That by itself would not be the problem. The problem is that even if I follow all next links and collect all the results it's hardly ever complete. Depending on the maxResults parameter different files are missing. Is really nobody else seeing that? We've a workaround and keep all file-ids in our database and fetch the missing ones one-by-one. But that's hundreds of extra requests and lot's of overhead in our code. – Lenau Aug 07 '13 at 06:44
  • Is there an open bug for this that we can monitor? – pinoyyid Aug 23 '13 at 08:13
  • @burcu, has this bug been fixed yet? Is anyone else still experiencing this bug? – kiwidrew Dec 10 '15 at 08:05

1 Answers1

0

Are there any difference in the files belonging to "shared to me" and own files/folders, was the issue for me ? The way it is presented in Google Drive was not the same result I got when searching without the correct flags.

I found out when I did this file list with all the folders, that I did have to include from where the search scope of files should be. - Include deleted files - Include shared to me files

Andreas Mattisson
  • 1,051
  • 2
  • 19
  • 39