feat(sea): execute statements asynchronously (submit + poll) for Thrift runAsync parity#406
Open
msrathore-db wants to merge 1 commit into
Open
Conversation
This was referenced May 31, 2026
…ft parity The SEA backend's executeStatement used the blocking napi executeStatement (kernel polled to terminal internally), so SeaOperationBackend.status() synthesized a constant FINISHED and a long-running query could not be cancelled from JS until it returned. The Thrift backend always runs async (runAsync: true): it submits, gets a pending operation handle, and polls getOperationStatus to terminal during waitUntilReady. This brings the SEA path to the same model: - SeaSessionBackend.executeStatement now calls the napi submitStatement (kernel wait_timeout=0s), receiving a pending AsyncStatement handle. Metadata methods keep the blocking statement path (already terminal). - SeaOperationBackend supports both shapes: for the async path, waitUntilReady() polls status() to terminal (firing the progress callback each tick, like the Thrift getOperationStatus loop), then materialises the result stream via awaitResult(); status() reports the real Pending/Running/Succeeded/Failed state; a Failed statement surfaces the kernel's typed SQL-error envelope via awaitResult()'s rejection. - The JS-side poll loop (not a single blocking awaitResult) keeps cancel() responsive: the kernel AsyncStatement serialises its methods behind one mutex, so a single in-flight awaitResult() would queue cancel() behind it for the whole query; polling status() releases the mutex between ticks. - SeaNativeLoader gains the typed async surface (submitStatement, SeaNativeAsyncStatement, SeaNativeAsyncResultHandle, SeaNativeStatementStatus). Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <[email protected]>
b4ab849 to
1a327fd
Compare
This was referenced May 31, 2026
Closed
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Brings the SEA execution path to the Thrift backend's always-async model (
runAsync: true). Previously the SEA backend used the blocking napiexecuteStatement(the kernel polled to terminal internally), so:SeaOperationBackend.status()synthesized a constantFINISHED_STATE— it never reflected the real server state.execute()blocked the call.The Thrift backend submits, gets a pending operation handle back immediately, and polls
getOperationStatusto a terminal state insidewaitUntilReady(). This PR makes SEA do the same.Changes
SeaSessionBackend.executeStatement→ calls the napisubmitStatement(kernelwait_timeout=0s), receiving a pendingAsyncStatement. Metadata methods keep the blocking statement path (the kernel returns those already-terminal).SeaOperationBackendnow accepts either{ asyncStatement }(query path) or{ statement }(metadata path):waitUntilReady()pollsstatus()to a terminal state, firing the progress callback each tick (mirrors the ThriftgetOperationStatusloop,delay(100)), then materialises the result stream viaawaitResult().status()maps the kernelStatementStatus(Pending/Running/Succeeded/Failed/Cancelled/Closed) to the matchingTOperationState.Failedstatement surfaces the kernel's typed SQL-error envelope viaawaitResult()'s rejection (status()only carries the variant name).SeaNativeLoadergains the typed async surface:submitStatement,SeaNativeAsyncStatement,SeaNativeAsyncResultHandle,SeaNativeStatementStatus.Why a JS-side poll loop (not a single blocking
awaitResult)The kernel
AsyncStatementserialises its methods behind one tokio mutex. A single in-flightawaitResult()would hold that mutex for the entire query and queuecancel()behind it — so cancel couldn't interrupt a running query. Pollingstatus()releases the mutex between ticks, leaving gaps forcancel()to land. This is what restores mid-run cancel parity.Depends on
Kernel PR databricks/databricks-sql-kernel#76 (
submitStatement+AsyncStatementnapi surface). The native binding must be built from a kernel that includes #76.Downstream fixes / reviewer note
FINISHED_STATEor cannot cancel a long-running query mid-flight, that bug is fixed here.flatbuffers, one-pass IPC duration rewrite, cancel/close local-state rollback on native RPC failure, closed fetch →OperationStateError(Closed), and Arrow duration rewriter cleanup.sessionId/statementIdtyping where available and rejects unsupported per-statementuseCloudFetchinstead of silently dropping it.Testing
SeaOperationBackend.test.ts(poll-to-terminal with per-tick callback, realRUNNINGstatus,Failed→ kernel error, mid-run cancel →OperationStateError).range(1e11) CROSS JOIN range(1000)mid-flight — both backends report mid-runoperationState = 2(RUNNING) and throw the identicalOperationStateError: The operation was canceled by a client. Basic SELECT and a 5000-row multi-batch fetch also verified end-to-end on the async path.Supersedes
PR #394 (
feat(sea): type async-execute napi surface + pecotesting e2e), which only typed the surface on the earlier M0 backend and did not wire the async lifecycle.This pull request and its description were written by Isaac.