The Common Mistake with `gh api --paginate --slurp --jq 'length'`

Many developers reach for gh api --paginate --slurp --jq 'length' when they need to count issues or pull requests in a GitHub repository. It seems intuitive: paginate through all results, slurp them into a single list, and then count the elements. The problem is, this command doesn't count issues or pull requests. It counts the number of API response pages returned during the operation. This misunderstanding, prevalent enough to appear in widely-read tutorials, leads to inaccurate counts and wasted effort.

I encountered this while developing pagination examples for a new toolkit. The correction felt necessary, especially after realizing a previously published lesson of mine contained this exact misconception, attracting significant search traffic. Understanding the precise behavior of --slurp and --paginate is key to avoiding this pitfall.

Understanding `--slurp` and `--paginate`

The GitHub CLI's help documentation for gh api (version 2.100.0) clarifies the role of --slurp:

Each page is a separate JSON array or object. Pass --slurp to wrap all pages of JSON arrays or objects into an outer JSON array.

Without --slurp, the --paginate flag causes gh api to issue a separate API request for each page of results. Your --jq filter is then applied independently to each of these individual page responses. This means if you're looking for a count, you'd get a count for each page, not a total.

When --slurp is added to the mix, gh api modifies this behavior. Instead of processing each page's JSON response individually, it collects all the responses and bundles them into a single, outer JSON array. The --jq 'length' filter is then applied to this single, outer array. The result is the total number of elements within that outer array. If each element in the JSON response represents an issue or a pull request, and each page contains multiple such elements, then the length of this slurped array *would* correctly represent the total count of issues or pull requests.

However, the critical detail is what constitutes an 'element' in the context of the GitHub API's paginated responses. For many endpoints, like listing issues or pull requests, the API returns an array of issue/PR objects on each page. When --slurp wraps these pages, the outer array contains the *pages themselves* as elements, or rather, the *contents of each page* are concatenated into a single array. The length filter, applied to this consolidated array, counts the number of items directly within it. If these items are indeed issues or pull requests, then length works as intended.

The Real Problem: API Response Structure

The issue arises because not all API endpoints return a simple array of objects at the top level of their paginated responses. Some might return a JSON object containing metadata and a nested array of the actual items. For instance, a common response structure for listing resources might look like this:

{
  "total_count": 1234,
  "incomplete_results": false,
  "items": [
    { "id": 1, "title": "..." },
    { "id": 2, "title": "..." },
    // ... more items
  ]
}

In this scenario, gh api --paginate --slurp --jq 'length' would operate on the entire JSON object returned after slurping. If the --jq filter is simply length, it counts the top-level keys of this object (e.g., "total_count", "incomplete_results", "items"), which is invariably a small, fixed number—not the count of issues.

The solution is to target the specific array containing the items. Using the example above, the correct --jq filter would be '.items | length'. This first navigates into the items array using the filter .items, and then applies the length filter to that specific array.

The surprising detail here is not the complexity of the GitHub API, but how a seemingly simple command can lead to such a fundamental misinterpretation of its output, especially when the command itself appears to be a standard pattern.

Correctly Counting Issues and Pull Requests

To accurately count issues or pull requests using the GitHub CLI, you must inspect the API response structure for the specific endpoint you are querying and adjust your --jq filter accordingly.

For Issues:

The endpoint for listing repository issues is typically /repos/{owner}/{repo}/issues. This endpoint returns a JSON object where the actual issues are found in the items key when searched, or directly as an array when listed. For a simple list of issues (not search results), the response is often a direct array of issue objects. In this case, gh api --paginate /repos/{owner}/{repo}/issues --jq 'length' *would* work correctly, as --slurp is not needed and the top-level response *is* the array of issues.

However, for searching issues (e.g., using the /search/issues endpoint), the response structure includes metadata. The correct command to count issues found via search would be:

gh api --paginate /search/issues --jq '.items | length'

Note the absence of --slurp. When --paginate is used without --slurp, the --jq filter is applied to each page. The GitHub API's search endpoint returns results with a structure like the one described above, where items contains the array of issues. By piping the output of each page's .items array to length, and then summing these lengths implicitly (as gh does when applying the filter page by page), you get the total count.

For Pull Requests:

Pull requests are treated as a subset of issues by the GitHub API. The endpoint /repos/{owner}/{repo}/pulls returns a direct array of pull request objects. Therefore, to count pull requests, the command is:

gh api --paginate /repos/{owner}/{repo}/pulls --jq 'length'

Here, --slurp is not needed, and --jq 'length' correctly counts the number of pull request objects in the paginated response.

What This Means for Developers

If you've used the gh api --paginate --slurp --jq 'length' pattern to count issues or pull requests, your counts have likely been inaccurate. The correct approach depends on the specific API endpoint and its response structure. For simple listings, --paginate with --jq 'length' suffices. For search results, you need to target the nested items array, and --slurp is often unnecessary or even detrimental when used incorrectly with length.

This highlights the importance of understanding the underlying API contracts. Relying on common command patterns without verifying their behavior against specific API responses can lead to subtle but significant errors. Always consult the GitHub API documentation for the exact structure of the responses you are working with.