Skip to content

feat(sea): execute statements asynchronously (submit + poll) for Thrift runAsync parity#406

Open
msrathore-db wants to merge 1 commit into
msrathore/sea-interval-getinfo-parityfrom
msrathore/sea-async-execute-impl
Open

feat(sea): execute statements asynchronously (submit + poll) for Thrift runAsync parity#406
msrathore-db wants to merge 1 commit into
msrathore/sea-interval-getinfo-parityfrom
msrathore/sea-async-execute-impl

Conversation

@msrathore-db
Copy link
Copy Markdown
Contributor

@msrathore-db msrathore-db commented May 31, 2026

What

Brings the SEA execution path to the Thrift backend's always-async model (runAsync: true). Previously the SEA backend used the blocking napi executeStatement (the kernel polled to terminal internally), so:

  • SeaOperationBackend.status() synthesized a constant FINISHED_STATE — it never reflected the real server state.
  • A long-running query could not be cancelled from JS until it returned, because execute() blocked the call.

The Thrift backend submits, gets a pending operation handle back immediately, and polls getOperationStatus to a terminal state inside waitUntilReady(). This PR makes SEA do the same.

Changes

  • SeaSessionBackend.executeStatement → calls the napi submitStatement (kernel wait_timeout=0s), receiving a pending AsyncStatement. Metadata methods keep the blocking statement path (the kernel returns those already-terminal).
  • SeaOperationBackend now accepts either { asyncStatement } (query path) or { statement } (metadata path):
    • waitUntilReady() polls status() to a terminal state, firing the progress callback each tick (mirrors the Thrift getOperationStatus loop, delay(100)), then materialises the result stream via awaitResult().
    • status() maps the kernel StatementStatus (Pending/Running/Succeeded/Failed/Cancelled/Closed) to the matching TOperationState.
    • A Failed statement surfaces the kernel's typed SQL-error envelope via awaitResult()'s rejection (status() only carries the variant name).
  • SeaNativeLoader gains the typed async surface: submitStatement, SeaNativeAsyncStatement, SeaNativeAsyncResultHandle, SeaNativeStatementStatus.

Why a JS-side poll loop (not a single blocking awaitResult)

The kernel AsyncStatement serialises its methods behind one tokio mutex. A single in-flight awaitResult() would hold that mutex for the entire query and queue cancel() behind it — so cancel couldn't interrupt a running query. Polling status() releases the mutex between ticks, leaving gaps for cancel() to land. This is what restores mid-run cancel parity.

Depends on

Kernel PR databricks/databricks-sql-kernel#76 (submitStatement + AsyncStatement napi surface). The native binding must be built from a kernel that includes #76.

Downstream fixes / reviewer note

Testing

  • 215 SEA unit tests pass, including a new async-lifecycle block in SeaOperationBackend.test.ts (poll-to-terminal with per-tick callback, real RUNNING status, Failed → kernel error, mid-run cancel → OperationStateError).
  • Live A/B against a real warehouse (Thrift vs SEA, identical query): cancelling a long-running range(1e11) CROSS JOIN range(1000) mid-flight — both backends report mid-run operationState = 2 (RUNNING) and throw the identical OperationStateError: The operation was canceled by a client. Basic SELECT and a 5000-row multi-batch fetch also verified end-to-end on the async path.

Supersedes

PR #394 (feat(sea): type async-execute napi surface + pecotesting e2e), which only typed the surface on the earlier M0 backend and did not wire the async lifecycle.

This pull request and its description were written by Isaac.

…ft parity

The SEA backend's executeStatement used the blocking napi executeStatement
(kernel polled to terminal internally), so SeaOperationBackend.status()
synthesized a constant FINISHED and a long-running query could not be
cancelled from JS until it returned. The Thrift backend always runs
async (runAsync: true): it submits, gets a pending operation handle, and
polls getOperationStatus to terminal during waitUntilReady.

This brings the SEA path to the same model:

- SeaSessionBackend.executeStatement now calls the napi submitStatement
  (kernel wait_timeout=0s), receiving a pending AsyncStatement handle.
  Metadata methods keep the blocking statement path (already terminal).
- SeaOperationBackend supports both shapes: for the async path,
  waitUntilReady() polls status() to terminal (firing the progress
  callback each tick, like the Thrift getOperationStatus loop), then
  materialises the result stream via awaitResult(); status() reports the
  real Pending/Running/Succeeded/Failed state; a Failed statement surfaces
  the kernel's typed SQL-error envelope via awaitResult()'s rejection.
- The JS-side poll loop (not a single blocking awaitResult) keeps cancel()
  responsive: the kernel AsyncStatement serialises its methods behind one
  mutex, so a single in-flight awaitResult() would queue cancel() behind
  it for the whole query; polling status() releases the mutex between ticks.
- SeaNativeLoader gains the typed async surface (submitStatement,
  SeaNativeAsyncStatement, SeaNativeAsyncResultHandle, SeaNativeStatementStatus).

Co-authored-by: Isaac
Signed-off-by: Madhavendra Rathore <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant