Leveraging conditional HTTP requests and Octokit hooks to avoid hitting rate limits against the GitHub REST API
As you may know, the GitHub API enforces certain rate limits to prevent abuse and denial-of-service attacks. So as an application developer, that's something you need to account for and handle in your applications interacting with GitHub. I would even recommend keeping this in mind during your very first design/architectural decisions.
Recently, I had to work on a TypeScript Backend application that needed to interact with several GitHub REST API endpoints in order to fetch data and return that in response to the end-user. These calls would be authenticated and sometimes even be made in various loops. The calls to GitHub were being done via the official Octokit client SDK libraries.
Surely, the GitHub GraphQL API could have certainly helped here, but note that it is also subject to rate limiting though enforced a bit differently from their REST API.
The GitHub documentation provides several relevant tips (link) on how to handle rate limits, but I wanted to shed light here on a specific technique I leveraged without adding too much overhead to the existing code: conditional requests coupled to Octokit hooks.
Without further ado, let's see this in action by quickly recalling what HTTP Conditional Requests are. As usual, the whole sample project is available for you to play with. This time, it is available at StackBlitz://conditional-requests-and-octokit-hooks-rate-limits.
Conditional HTTP Requests
In a nutshell, conditional HTTP requests allow an HTTP client to make requests to a server but receive a response only if certain conditions are met.
The key point here is that the server would return up-to-date data only if the requested data has changed since it was last requested. Otherwise, nothing is returned and it is up to the client to cache the previous response somewhere and return it accordingly.
In essence, these requests enable a client to check whether a cached version of a resource is still valid before retrieving it again, enhancing efficiency and ensuring data freshness.
Typical conditions include:
If-Modified-Since: the server responds only if the resource has been modified since the specified date.If-None-Match: the server responds only if the resource’s currentETagheader (an identifier in the request header representing the resource's state) doesn’t match the one provided by the client.
Some of the benefits of leveraging conditional HTTP requests include avoidance of rate limits or redundant downloads, saving bandwidth, and reducing server load, which is especially valuable in high-traffic applications.
Tutorial
Let's walk through a simple example project, which we can initialize using NPM:
mkdir my-octokit-project && cd my-octokit-project
npm init -y
npm install typescript --save-dev
npx tsc --initNow let's add the Octokit client libraries as dependencies to our sample project:
npm install @octokit/core
npm install @octokit/rest
npm install @types/node --save-dev
npm install @octokit/types --save-dev
Now we can proceed to creating our sample entry file under the src folder:
import { Octokit } from '@octokit/rest';
export class GHClient {
private readonly octokit: Octokit;
private readonly ghResponseCache: Map<string, any>;
constructor() {
this.ghResponseCache = new Map<string, any>();
this.octokit = new Octokit({
// purposely unauthenticated call, to hit rate limits faster
// auth: 'your-personal-access-token',
});
}
async fetchGHData(
user: string
): Promise<{ userOrgs: string[]; orgRepos: string[] }> {
// just a simple test illustrating multiple calls that might happen against the GitHub API
const userOrgs = new Set<string>();
const orgRepos = new Set<string>();
for (let i = 0; i < 100; i++) {
const orgsResp = await this.octokit.rest.orgs.listForUser({
username: user,
page: 1,
per_page: 20,
});
for (const org of orgsResp.data) {
userOrgs.add(org.login);
const repoResp = await this.octokit.rest.repos.listForOrg({
org: org.login,
page: 1,
per_page: 10,
});
repoResp.data.forEach(async (r) => {
orgRepos.add(`${r.owner.login}/${r.name}`);
await this.octokit.rest.repos.get({
owner: r.owner.login,
repo: r.name,
});
});
}
}
return {
userOrgs: Array.from(userOrgs.values()),
orgRepos: Array.from(orgRepos.values()),
};
}
// --- Truncated, but we can imagine other methods here
}
const ghClient = new GHClient();
ghClient
.fetchGHData('rm3l')
.then((data) => console.log(data))
.catch((err) => console.log(`[WARN] error: ${err}`));
In the overly-simplified sample above, we are encapsulating an Octokit client in our own GHClient class and adding some methods relevant to our business case. Let's compile and run it. At some point, we might end up hitting rate limits, like so:
❯ npx tsc && node dist/index.js
GET /repos/asdf-community/asdf-php - 403 with id UNKNOWN in 37ms
RequestError [HttpError]: API rate limit exceeded for x.y.z.t. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.) - https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting
at new RequestError (file:///home/projects/stackblitz-starters-mv73mt/node_modules/@octokit/request-error/dist-src/index.js:29:5)
at fetchWrapper (file:///home/projects/stackblitz-starters-mv73mt/node_modules/@octokit/request/dist-bundle/index.js:134:11) {
status: 403,
request: {
method: 'GET',
url: 'https://api.github.com/repos/asdf-community/asdf-php',
headers: {
accept: 'application/vnd.github.v3+json',
'user-agent': 'octokit-rest.js/21.0.2 octokit-core.js/6.1.2 Node.js/18.20.3 (linux; x64)'
},
request: { hook: [Function: bound bound register] }
},
response: {
url: 'https://api.github.com/repos/asdf-community/asdf-php',
status: 403,
headers: {
'content-length': '279',
'content-type': 'application/json; charset=utf-8',
'x-github-media-type': 'github.v3; format=json',
'x-ratelimit-limit': '60',
'x-ratelimit-remaining': '0',
'x-ratelimit-reset': '1730675727',
'x-ratelimit-resource': 'core',
'x-ratelimit-used': '60'
},
data: {
message: "API rate limit exceeded for x.y.z.t. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)",
documentation_url: 'https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting'
}
}
}Example output with rate limit errors
To avoid hitting rate limits from the GitHub API, we are now going to modify our GHClient class by registering hooks that allow to manipulate any requests/responses sent/received via the Octokit client. This caches responses that have not changed in a really simple map, but a more realistic example could leverage a different caching mechanism (like Least Recently Used - LRU) to better harness the memory consumption.
diff --git a/src/index.ts b/src/index.ts
index 02bf785..3ca4929 100644
--- a/src/index.ts
+++ b/src/index.ts
@@ -2,11 +2,73 @@ import { Octokit } from '@octokit/rest';
export class GHClient {
private readonly octokit: Octokit;
+ private readonly ghResponseCache: Map<string, any>;
constructor() {
+ this.ghResponseCache = new Map<string, any>();
this.octokit = new Octokit({
//auth: 'your-personal-access-token',
});
+ this.registerHooks();
+ }
+
+ private registerHooks() {
+ const tryReplacingPlaceholdersInUrl = (options: any): string => {
+ // options.url might contain placeholders, like '/api/orgs/{org}/repos'
+ // => need to replace them with their actual values to not get colliding keys
+ return options.url.replace(/{([^}]+)}/g, (_: any, key: any) => {
+ // Replace with actual value, or leave unchanged if not found
+ return options[key] ?? `{${key}}`;
+ });
+ };
+ const getCacheKey = (options: any): string =>
+ `${options.method}--${tryReplacingPlaceholdersInUrl(options)}`;
+
+ this.octokit.hook.before('request', async (options) => {
+ if (!options.headers) {
+ options.headers = {
+ accept: 'application/json',
+ 'user-agent': 'com.example/my-awesome-user-agent',
+ };
+ }
+ // Use ETag from in-memory cache if available
+ const cacheKey = getCacheKey(options);
+ const existingEtag = this.ghResponseCache.get(cacheKey)?.etag;
+ if (existingEtag) {
+ options.headers['If-None-Match'] = existingEtag;
+ } else {
+ console.log(`[DEBUG] cache miss for key "${cacheKey}"`);
+ }
+ });
+
+ this.octokit.hook.after('request', async (response, options) => {
+ console.log(
+ `[DEBUG] [GH API] ${options.method} ${tryReplacingPlaceholdersInUrl(
+ options
+ )}: ${response.status}`
+ );
+ // A successful response means that the resource has changed
+ // => so update the in-memory cache
+ const cacheKey = getCacheKey(options);
+ this.ghResponseCache.set(cacheKey, {
+ etag: response.headers.etag,
+ ...response,
+ });
+ });
+
+ this.octokit.hook.error('request', async (error: any, options) => {
+ console.log(
+ `[DEBUG] [GH API] ${options.method} ${tryReplacingPlaceholdersInUrl(
+ options
+ )}: ${error.status}`
+ );
+ if (error.status !== 304) {
+ throw error;
+ }
+ // "304 Not Modified" means that the resource hasn't changed,
+ // and we should have a version of it in the cache
+ return this.ghResponseCache.get(getCacheKey(options));
+ });
}
async fetchGHData(Note that this does not modify the existing calls to the GitHub API; instead, it registers in a single place a set of hooks that allow us to customize the Octokit’s request lifecycle.
Now if we run this again, we should probably not hit the rate limit errors anymore:
❯ npx tsc && node dist/index.js
[DEBUG] cache miss for key "GET--/users/rm3l/orgs"
[DEBUG] [GH API] GET /users/rm3l/orgs: 200
[DEBUG] cache miss for key "GET--/orgs/maoni-app/repos"
[DEBUG] [GH API] GET /orgs/maoni-app/repos: 200
[DEBUG] cache miss for key "GET--/orgs/asdf-community/repos"
[DEBUG] [GH API] GET /orgs/asdf-community/repos: 200
[DEBUG] cache miss for key "GET--/orgs/devfile/repos"
[DEBUG] [GH API] GET /orgs/devfile/repos: 200
[DEBUG] cache miss for key "GET--/orgs/ododev/repos"
[DEBUG] [GH API] GET /orgs/ododev/repos: 200
[DEBUG] cache miss for key "GET--/orgs/lemra-org/repos"
[DEBUG] [GH API] GET /orgs/lemra-org/repos: 200
[...]
[DEBUG] [GH API] GET /users/rm3l/orgs: 304
[DEBUG] [GH API] GET /orgs/asdf-community/repos: 304
[DEBUG] [GH API] GET /orgs/devfile/repos: 304
[DEBUG] [GH API] GET /orgs/ododev/repos: 304
[DEBUG] [GH API] GET /orgs/lemra-org/repos: 304
[...]Why this works?
Under the hood, the Octokit hooks build upon the before-after-hook library.
The after hook is called upon a successful response from GitHub, and in such case we store the response ETag and overall response object in our cache.
The before hook, as the name suggests, is called before any request that is about to be sent to the GitHub API. In this hook, we set the If-None-Match request header to the previous ETag header we had cached for this specific request, if any. The GitHub API will leverage the value of this header and will return a 304 Not Modified response with nothing in the body if the response had not changed since this ETag. And a 304 Not Modified response does not count towards our rate limit usage.
Otherwise, if the resource had changed, the GitHub API will just return the updated data along with a new ETag response header, which will count towards our rate limit usage.
Finally, the error hook is where we actually handle errors. Generally speaking, statuses other than a 2xx (like304 Not Modified) are treated as errors. For a 304 Not Modified, we assume that the response was already cached previously and just return what we have in the cache for this request.
Wrapping Up
As we have seen in this blog post, the Octokit client library hooks provide an interesting approach to limit the risk of hitting rate limits from the GitHub API in a TypeScript or JavaScript application. I would still recommend the official Best practices for using the GitHub REST API for your awareness. As usual, feel to share your thoughts in the comments.