Skip to content

fix(bigtable): pack unpacked error details in _get_status and harden batcher error handling - #18770

Open
mutianf wants to merge 5 commits into
googleapis:mainfrom
mutianf:fix-get-status-unpacked-details
Open

mutianf wants to merge 5 commits into
googleapis:mainfrom
mutianf:fix-get-status-unpacked-details

Conversation

@mutianf

@mutianf mutianf commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

Summary

  • _get_status unpacked protobuf error details: google.api_core.exceptions.from_grpc_error unpacks google.protobuf.any_pb2.Any messages in grpc-status-details-bin into concrete google.rpc.error_details_pb2 protobuf objects (ErrorInfo, RetryInfo, DebugInfo, etc.) in exc.details. Previously, _get_status passed details=exc.details directly to status_pb2.Status(...), raising TypeError: Parameter to initialize message field must be dict or instance of same class: expected <class 'google.protobuf.any_pb2.Any'> got ... whenever a MutateRows RPC failed with rich gRPC error details. _get_status now normalizes each item in exc.details, keeping any_pb2.Any messages and packing unpacked google.protobuf.message.Message instances into any_pb2.Any.
  • MutationsBatcherAsync / MutationsBatcher error handling:
    • In _execute_mutate_rows, only compute _get_statuses_from_mutations_exception_group and Status lists when self._user_batch_completed_callback is not None (falling back to UNKNOWN statuses if an unexpected non-MutationsExceptionGroup exception occurs).
    • In MutationsBatcherAsync._wait_for_batch_results, move await task inside the try: block so unexpected task exceptions in async mode are caught and wrapped in FailedMutationEntryError rather than aborting _flush_internal before _add_exceptions (which previously caused batcher.close() to silently drop failed mutations).
  • _get_error_type status code validation: core_exceptions.from_grpc_status(call_code, "") maps 0 (grpc.StatusCode.OK) and unmapped integers to the base class GoogleAPICallError, which caused retries.if_exception_type to match and retry every GoogleAPICallError subclass. _get_error_type now raises ValueError for 0 / grpc.StatusCode.OK, booleans, and any unmapped status code that falls back to base GoogleAPICallError.

Test plan

  • Added unit tests in tests/unit/data/test__helpers.py for _get_status packing unpacked protobuf details and _get_error_type rejecting invalid status codes (0, grpc.StatusCode.OK, booleans, and unmapped codes).
  • Added unit tests in tests/unit/data/_async/test_mutations_batcher.py (and regenerated tests/unit/data/_sync_autogen/test_mutations_batcher.py) for _execute_mutate_rows callback fallback on unexpected errors and _wait_for_batch_results catching unexpected task exceptions.
  • Verified test_sync_up_to_date.py and full packages/google-cloud-bigtable/tests/unit suite pass.

@mutianf
mutianf requested a review from a team as a code owner October 19, 2026 18:26

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request optimizes the mutations batcher by deferring status list creation until a user callback is present, and wraps the async task awaiting in a try-catch block to handle unexpected exceptions. It also updates error helper functions to correctly pack unpacked error details into protobuf Any messages and validate gRPC error status codes, accompanied by comprehensive unit tests. The review feedback suggests a valuable simplification in _wait_for_batch_results to directly assign the awaited task result in async mode instead of calling task.result() redundantly.

…RetryExceptionGroup in legacy batcher

Change-Id: Id9737b9695e281f340fce06712362ee296ae71af
… TypeError and batcher fixes

Change-Id: Iba6a2a1085355d70b436563c48b66cc7fd9082a7
…itting isinstance check

Change-Id: I6cadc96645514c90bed567ce4bddd9e39c1a8cc8
@mutianf

mutianf commented Oct 19, 2026

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces several robustness improvements to the Bigtable mutations batcher and helper utilities. Key changes include extracting the terminal exception from a RetryExceptionGroup in the batcher, refactoring user batch completion callback execution to properly handle synchronous and asynchronous contexts, moving task awaiting inside try-except blocks to catch unexpected task exceptions, packing unpacked error details into status_pb2.Status objects, and validating gRPC error status codes in _get_error_type. Corresponding unit tests have been added to verify these behaviors. Feedback on the changes suggests defaulting exc.message to an empty string when constructing status_pb2.Status to prevent potential TypeError exceptions if the message is None.

details.append(packed)
return status_pb2.Status(
code=grpc_code.value[0],
message=exc.message,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If exc.message is None, passing it directly to status_pb2.Status will raise a TypeError because protobuf string fields do not accept None. We should default it to an empty string using exc.message or "" to ensure robustness.

Suggested change
message=exc.message,
message=exc.message or "",

…result in async mode

Change-Id: I680fe5cb809dcbd01e293d9318d576e0df49d189

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant