This repository was archived by the owner on Jan 9, 2023. It is now read-only.
Repository navigation
This repository was archived by the owner on Jan 9, 2023. It is now read-only.
Export more than the first 1000 issues #42
Description
I ran this on microsoft/TypeScript and was surprised to see only a 1.6 MB file produced, since I know we have much more content than that. The generated JSON, though, only includes 1,000 issues, so it misses the other ~95% of the issues to have crossed our repo.
Activity
Chocrates commentedon May 19, 2021
Thanks for the report @RyanCavanaugh ! That seems bad! I will dig in to this and see what is going on.
Chocrates commentedon May 19, 2021
I am able to repro this:
./bin/run search:issues --token <token> --owner microsoft --repo typescript --format JSON --since=2020-06-01 > typescript.jsonChocrates commentedon May 19, 2021
Looks like the GraphQL query gets to page 10 and then can't find the next page (
hasNextPageis false). Debugging this to see what is happening.RyanCavanaugh commentedon May 19, 2021
The Search API is documented to only return the first 1,000 results
I've written a similar tool for bulk export and AFAIK the only alternative is to go through the Issues graph instead of the Search graph.
Chocrates commentedon May 25, 2021
I opened #45 to discuss how we want to solve this issue, since it will require a bit of re-architecting.