feat(gax): support resumable uploads by feywind · Pull Request #9287 · googleapis/google-cloud-node

feywind · 2026-09-10T22:08:00Z

Adds the client-side implementation of the resumable upload protocol to google-gax.

What's here

ResumableUploadDescriptor / ResumableUploadSession and the resumableUploadStub that generated clients wire into createApiCall
resumableSourceFromFile, a seekable upload source backed by a local file
CallOptions.resumableUpload, the transport context generated clients pass to the stub
exports from index, fallback and descriptor, plus user documentation in client-libraries.md
unit tests and a hermetic system test (real HTTP server, no credentials) covering the state machine, transient retries, recovery from state mismatches and resume from a saved session URL

onProgress callbacks may return void; the previous signature rejected the documented usage, which only logs progress.

Verification

npx tsc -p . — no errors in the touched files
npx mocha build/test/unit — 406 passing
npx mocha build/test/system-test/resumableUpload.js — 2 passing

Nothing in existing behaviour changes: the new session is only reachable through the new descriptor and stub.

Related to: #9283

Add the client-side implementation of the resumable upload protocol: - ResumableUploadDescriptor and ResumableUploadSession, plus the resumableUploadStub that generated clients wire into createApiCall - resumableSourceFromFile, a seekable source backed by a local file - CallOptions.resumableUpload carrying the transport context that generated clients pass to the stub - exports from index, fallback and descriptor, and client-libraries docs - unit and hermetic system tests covering the state machine, transient retries, recovery from state mismatches and resume from a saved URL The onProgress callback may return void; the documented usage logs progress without returning a value.

gemini-code-assist

Code Review

This pull request introduces support for the resumable upload protocol in the Google API Extensions (gax) library, adding a state machine to manage chunked uploads, retries, and session recovery, along with corresponding tests and documentation. Feedback highlights a critical runtime crash in the HTTP request handling due to an incorrect response type cast, which also requires updating mock responses in unit and system tests. Additionally, the host/port parsing regex should be improved to support IPv6 addresses, and manual edits to the auto-generated markdown documentation should be moved upstream to the generator templates.

CI type-checks this package with the repository root TypeScript (^5.8.3, currently 5.9.3) rather than the 5.8.3 pinned here, which tightens the Buffer generics: - annotate the transmission buffer as Buffer instead of letting Buffer.alloc() narrow it to Buffer<ArrayBuffer>, which readNextChunk() and skipBytes() cannot assign to - copy forwarded bodies into Uint8Array.from() in the system test, since fetch()'s BodyInit does not accept Buffer<ArrayBufferLike> - merge the two 'fs' imports in resumableSourceFromFile.ts Found by the monorepo linter (bin/linter.mjs) on googleapis#9287.

- The client-libraries.md examples passed `uploadStream: dataStream`, but the session takes `uploadSource` (a `ResumableSource`) and `dataStream` was never defined, so the snippets would not compile. Use `client.getResumableSource()` in both examples, and correct the helper name to `ResumableUploadSession`. - `DEFAULT_UPLOAD_RATE_BYTES_PER_MS` is bytes per millisecond but held `5 * 1024 * 1024`, i.e. ~5 GiB/s rather than the documented ~5 MiB/s. Since `computeGlobalDeadlineMs` combines the scaled value with `Math.max` against the 10 minute default, size-based scaling could not engage below a ~3 TB payload. Express the documented rate per millisecond instead.

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

feywind · 2026-09-21T22:08:57Z

Santi was going to look at this one, I think, but I'll bug you if that changes.

quirogas

Thanks for putting this together! Here are a few findings and nit suggestions from testing the state machine and generated client integration.

🤖 I have created a release *beep* *boop* --- <details><summary>google-gax: 6.6.0</summary> ## [6.6.0](google-gax-v6.5.0...google-gax-v6.6.0) (2026-09-23) ### Features * **gax:** Support resumable uploads ([#9287](#9287)) ([8116709](8116709)) </details> <details><summary>chat: 0.33.0</summary> ## [0.33.0](chat-v0.32.0...chat-v0.33.0) (2026-09-23) ### Features * Update API sources and regenerate ([#9427](#9427)) ([5cd945d](5cd945d)) </details> <details><summary>agentregistry: 0.5.0</summary> ## [0.5.0](agentregistry-v0.4.0...agentregistry-v0.5.0) (2026-09-23) ### Features * Update API sources and regenerate ([#9427](#9427)) ([5cd945d](5cd945d)) </details> <details><summary>google-cloud-bigtable-api: 0.8.0</summary> ## [0.8.0](google-cloud-bigtable-api-v0.7.0...google-cloud-bigtable-api-v0.8.0) (2026-09-23) ### Features * Update API sources and regenerate ([#9427](#9427)) ([5cd945d](5cd945d)) </details> <details><summary>geminidataanalytics: 0.13.0</summary> ## [0.13.0](geminidataanalytics-v0.12.0...geminidataanalytics-v0.13.0) (2026-09-23) ### Features * Update API sources and regenerate ([#9427](#9427)) ([5cd945d](5cd945d)) </details> <details><summary>lustre: 0.5.0</summary> ## [0.5.0](lustre-v0.4.0...lustre-v0.5.0) (2026-09-23) ### Features * Update API sources and regenerate ([#9427](#9427)) ([5cd945d](5cd945d)) </details> <details><summary>sql: 0.29.0</summary> ## [0.29.0](sql-v0.28.0...sql-v0.29.0) (2026-09-23) ### Features * Update API sources and regenerate ([#9427](#9427)) ([5cd945d](5cd945d)) </details> <details><summary>tasks: 7.2.0</summary> ## [7.2.0](tasks-v7.1.1...tasks-v7.2.0) (2026-09-23) ### Features * Update API sources and regenerate ([#9427](#9427)) ([5cd945d](5cd945d)) </details> <details><summary>vectorsearch: 0.13.0</summary> ## [0.13.0](vectorsearch-v0.12.0...vectorsearch-v0.13.0) (2026-09-23) ### Features * Update API sources and regenerate ([#9427](#9427)) ([5cd945d](5cd945d)) </details> <details><summary>marketing-platform-admin: 0.6.0</summary> ## [0.6.0](marketing-platform-admin-v0.5.0...marketing-platform-admin-v0.6.0) (2026-09-23) ### Features * Update API sources and regenerate ([#9427](#9427)) ([5cd945d](5cd945d)) </details> <details><summary>monitoring-dashboards: 5.1.0</summary> ## [5.1.0](monitoring-dashboards-v5.0.0...monitoring-dashboards-v5.1.0) (2026-09-23) ### Features * Update API sources and regenerate ([#9427](#9427)) ([5cd945d](5cd945d)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

Adds an end-to-end harness for the gapic-showcase `ResumableUploadService`, so the resumable upload client path can be exercised against a real server. It uses the resumable upload support from the `google-gax` PR in this series, so merge that one first. - checked-in generated client for `google.showcase.v1beta1.ResumableUploadService` - `sample.js`, which uploads a local file through a resumable session - `run.sh`, which downloads gapic-showcase, compiles the client against the local `google-gax` checkout and runs the sample - harness README plus a pointer from `test/README.md`, and a `.gitignore` entry for the generated protos This is test tooling only; nothing in the published package changes. Merge first: #9287 Related to: #9283

🤖 I have created a release *beep* *boop* --- <details><summary>google-gax: 6.6.0</summary> ## [6.6.0](googleapis/google-cloud-node@google-gax-v6.5.0...google-gax-v6.6.0) (2026-09-23) ### Features * **gax:** Support resumable uploads ([googleapis#9287](googleapis#9287)) ([8116709](googleapis@8116709)) </details> <details><summary>chat: 0.33.0</summary> ## [0.33.0](googleapis/google-cloud-node@chat-v0.32.0...chat-v0.33.0) (2026-09-23) ### Features * Update API sources and regenerate ([googleapis#9427](googleapis#9427)) ([5cd945d](googleapis@5cd945d)) </details> <details><summary>agentregistry: 0.5.0</summary> ## [0.5.0](googleapis/google-cloud-node@agentregistry-v0.4.0...agentregistry-v0.5.0) (2026-09-23) ### Features * Update API sources and regenerate ([googleapis#9427](googleapis#9427)) ([5cd945d](googleapis@5cd945d)) </details> <details><summary>google-cloud-bigtable-api: 0.8.0</summary> ## [0.8.0](googleapis/google-cloud-node@google-cloud-bigtable-api-v0.7.0...google-cloud-bigtable-api-v0.8.0) (2026-09-23) ### Features * Update API sources and regenerate ([googleapis#9427](googleapis#9427)) ([5cd945d](googleapis@5cd945d)) </details> <details><summary>geminidataanalytics: 0.13.0</summary> ## [0.13.0](googleapis/google-cloud-node@geminidataanalytics-v0.12.0...geminidataanalytics-v0.13.0) (2026-09-23) ### Features * Update API sources and regenerate ([googleapis#9427](googleapis#9427)) ([5cd945d](googleapis@5cd945d)) </details> <details><summary>lustre: 0.5.0</summary> ## [0.5.0](googleapis/google-cloud-node@lustre-v0.4.0...lustre-v0.5.0) (2026-09-23) ### Features * Update API sources and regenerate ([googleapis#9427](googleapis#9427)) ([5cd945d](googleapis@5cd945d)) </details> <details><summary>sql: 0.29.0</summary> ## [0.29.0](googleapis/google-cloud-node@sql-v0.28.0...sql-v0.29.0) (2026-09-23) ### Features * Update API sources and regenerate ([googleapis#9427](googleapis#9427)) ([5cd945d](googleapis@5cd945d)) </details> <details><summary>tasks: 7.2.0</summary> ## [7.2.0](googleapis/google-cloud-node@tasks-v7.1.1...tasks-v7.2.0) (2026-09-23) ### Features * Update API sources and regenerate ([googleapis#9427](googleapis#9427)) ([5cd945d](googleapis@5cd945d)) </details> <details><summary>vectorsearch: 0.13.0</summary> ## [0.13.0](googleapis/google-cloud-node@vectorsearch-v0.12.0...vectorsearch-v0.13.0) (2026-09-23) ### Features * Update API sources and regenerate ([googleapis#9427](googleapis#9427)) ([5cd945d](googleapis@5cd945d)) </details> <details><summary>marketing-platform-admin: 0.6.0</summary> ## [0.6.0](googleapis/google-cloud-node@marketing-platform-admin-v0.5.0...marketing-platform-admin-v0.6.0) (2026-09-23) ### Features * Update API sources and regenerate ([googleapis#9427](googleapis#9427)) ([5cd945d](googleapis@5cd945d)) </details> <details><summary>monitoring-dashboards: 5.1.0</summary> ## [5.1.0](googleapis/google-cloud-node@monitoring-dashboards-v5.0.0...monitoring-dashboards-v5.1.0) (2026-09-23) ### Features * Update API sources and regenerate ([googleapis#9427](googleapis#9427)) ([5cd945d](googleapis@5cd945d)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

Adds an end-to-end harness for the gapic-showcase `ResumableUploadService`, so the resumable upload client path can be exercised against a real server. It uses the resumable upload support from the `google-gax` PR in this series, so merge that one first. - checked-in generated client for `google.showcase.v1beta1.ResumableUploadService` - `sample.js`, which uploads a local file through a resumable session - `run.sh`, which downloads gapic-showcase, compiles the client against the local `google-gax` checkout and runs the sample - harness README plus a pointer from `test/README.md`, and a `.gitignore` entry for the generated protos This is test tooling only; nothing in the published package changes. Merge first: googleapis#9287 Related to: googleapis#9283

Adds the client-side implementation of the resumable upload protocol to `google-gax`. ### What's here - `ResumableUploadDescriptor` / `ResumableUploadSession` and the `resumableUploadStub` that generated clients wire into `createApiCall` - `resumableSourceFromFile`, a seekable upload source backed by a local file - `CallOptions.resumableUpload`, the transport context generated clients pass to the stub - exports from `index`, `fallback` and `descriptor`, plus user documentation in `client-libraries.md` - unit tests and a hermetic system test (real HTTP server, no credentials) covering the state machine, transient retries, recovery from state mismatches and resume from a saved session URL `onProgress` callbacks may return `void`; the previous signature rejected the documented usage, which only logs progress. ### Verification - `npx tsc -p .` — no errors in the touched files - `npx mocha build/test/unit` — 406 passing - `npx mocha build/test/system-test/resumableUpload.js` — 2 passing Nothing in existing behaviour changes: the new session is only reachable through the new descriptor and stub. Related to: #9283 --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

🤖 I have created a release *beep* *boop* --- <details><summary>google-gax: 6.6.0</summary> ## [6.6.0](google-gax-v6.5.0...google-gax-v6.6.0) (2026-09-23) ### Features * **gax:** Support resumable uploads ([#9287](#9287)) ([8116709](8116709)) </details> <details><summary>chat: 0.33.0</summary> ## [0.33.0](chat-v0.32.0...chat-v0.33.0) (2026-09-23) ### Features * Update API sources and regenerate ([#9427](#9427)) ([5cd945d](5cd945d)) </details> <details><summary>agentregistry: 0.5.0</summary> ## [0.5.0](agentregistry-v0.4.0...agentregistry-v0.5.0) (2026-09-23) ### Features * Update API sources and regenerate ([#9427](#9427)) ([5cd945d](5cd945d)) </details> <details><summary>google-cloud-bigtable-api: 0.8.0</summary> ## [0.8.0](google-cloud-bigtable-api-v0.7.0...google-cloud-bigtable-api-v0.8.0) (2026-09-23) ### Features * Update API sources and regenerate ([#9427](#9427)) ([5cd945d](5cd945d)) </details> <details><summary>geminidataanalytics: 0.13.0</summary> ## [0.13.0](geminidataanalytics-v0.12.0...geminidataanalytics-v0.13.0) (2026-09-23) ### Features * Update API sources and regenerate ([#9427](#9427)) ([5cd945d](5cd945d)) </details> <details><summary>lustre: 0.5.0</summary> ## [0.5.0](lustre-v0.4.0...lustre-v0.5.0) (2026-09-23) ### Features * Update API sources and regenerate ([#9427](#9427)) ([5cd945d](5cd945d)) </details> <details><summary>sql: 0.29.0</summary> ## [0.29.0](sql-v0.28.0...sql-v0.29.0) (2026-09-23) ### Features * Update API sources and regenerate ([#9427](#9427)) ([5cd945d](5cd945d)) </details> <details><summary>tasks: 7.2.0</summary> ## [7.2.0](tasks-v7.1.1...tasks-v7.2.0) (2026-09-23) ### Features * Update API sources and regenerate ([#9427](#9427)) ([5cd945d](5cd945d)) </details> <details><summary>vectorsearch: 0.13.0</summary> ## [0.13.0](vectorsearch-v0.12.0...vectorsearch-v0.13.0) (2026-09-23) ### Features * Update API sources and regenerate ([#9427](#9427)) ([5cd945d](5cd945d)) </details> <details><summary>marketing-platform-admin: 0.6.0</summary> ## [0.6.0](marketing-platform-admin-v0.5.0...marketing-platform-admin-v0.6.0) (2026-09-23) ### Features * Update API sources and regenerate ([#9427](#9427)) ([5cd945d](5cd945d)) </details> <details><summary>monitoring-dashboards: 5.1.0</summary> ## [5.1.0](monitoring-dashboards-v5.0.0...monitoring-dashboards-v5.1.0) (2026-09-23) ### Features * Update API sources and regenerate ([#9427](#9427)) ([5cd945d](5cd945d)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

Adds an end-to-end harness for the gapic-showcase `ResumableUploadService`, so the resumable upload client path can be exercised against a real server. It uses the resumable upload support from the `google-gax` PR in this series, so merge that one first. - checked-in generated client for `google.showcase.v1beta1.ResumableUploadService` - `sample.js`, which uploads a local file through a resumable session - `run.sh`, which downloads gapic-showcase, compiles the client against the local `google-gax` checkout and runs the sample - harness README plus a pointer from `test/README.md`, and a `.gitignore` entry for the generated protos This is test tooling only; nothing in the published package changes. Merge first: #9287 Related to: #9283

feywind mentioned this pull request Sep 10, 2026

test(gax): add showcase harness for resumable uploads #9288

Merged

gemini-code-assist Bot reviewed Sep 10, 2026

View reviewed changes

feywind and others added 6 commits September 10, 2026 18:36

Merge branch 'main' into resumable/gax

1fbdfd2

fix: count 409 as a cat 2 retry code

3402bc8

chore: update regex for ipv6 addresses

ab4c937

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Merge branch 'main' into resumable/gax

1f31294

feywind marked this pull request as ready for review September 21, 2026 21:20

feywind requested a review from a team as a code owner September 21, 2026 21:20

github-actions Bot requested a review from shivanee-p September 21, 2026 21:35

quirogas reviewed Sep 22, 2026

View reviewed changes

feywind mentioned this pull request Sep 22, 2026

test(gax): add showcase harness for resumable uploads #9416

Closed

fix(gax): address review comments on resumable upload session

d091888

quirogas approved these changes Sep 23, 2026

View reviewed changes

feywind merged commit 8116709 into googleapis:main Sep 23, 2026
50 checks passed

release-please Bot mentioned this pull request Sep 23, 2026

chore: release main #9428

Merged

feywind deleted the resumable/gax branch September 24, 2026 18:26

Navigation Menu

Repository navigation

feat(gax): support resumable uploads - #9287

feat(gax): support resumable uploads#9287
feywind merged 8 commits into
googleapis:mainfrom
feywind:resumable/gax

feywind commented Sep 10, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

feywind commented Sep 21, 2026

Uh oh!

quirogas left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

feywind commented Sep 10, 2026

What's here

Verification

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

feywind commented Sep 21, 2026

Uh oh!

quirogas left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants