Repository navigation
Conversation
…batcher error handling
There was a problem hiding this comment.
Code Review
This pull request optimizes the mutations batcher by deferring status list creation until a user callback is present, and wraps the async task awaiting in a try-catch block to handle unexpected exceptions. It also updates error helper functions to correctly pack unpacked error details into protobuf Any messages and validate gRPC error status codes, accompanied by comprehensive unit tests. The review feedback suggests a valuable simplification in _wait_for_batch_results to directly assign the awaited task result in async mode instead of calling task.result() redundantly.
…RetryExceptionGroup in legacy batcher Change-Id: Id9737b9695e281f340fce06712362ee296ae71af
… TypeError and batcher fixes Change-Id: Iba6a2a1085355d70b436563c48b66cc7fd9082a7
…itting isinstance check Change-Id: I6cadc96645514c90bed567ce4bddd9e39c1a8cc8
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces several robustness improvements to the Bigtable mutations batcher and helper utilities. Key changes include extracting the terminal exception from a RetryExceptionGroup in the batcher, refactoring user batch completion callback execution to properly handle synchronous and asynchronous contexts, moving task awaiting inside try-except blocks to catch unexpected task exceptions, packing unpacked error details into status_pb2.Status objects, and validating gRPC error status codes in _get_error_type. Corresponding unit tests have been added to verify these behaviors. Feedback on the changes suggests defaulting exc.message to an empty string when constructing status_pb2.Status to prevent potential TypeError exceptions if the message is None.
| details.append(packed) | ||
| return status_pb2.Status( | ||
| code=grpc_code.value[0], | ||
| message=exc.message, |
There was a problem hiding this comment.
…result in async mode Change-Id: I680fe5cb809dcbd01e293d9318d576e0df49d189
Summary
_get_statusunpacked protobuf error details:google.api_core.exceptions.from_grpc_errorunpacksgoogle.protobuf.any_pb2.Anymessages ingrpc-status-details-bininto concretegoogle.rpc.error_details_pb2protobuf objects (ErrorInfo,RetryInfo,DebugInfo, etc.) inexc.details. Previously,_get_statuspasseddetails=exc.detailsdirectly tostatus_pb2.Status(...), raisingTypeError: Parameter to initialize message field must be dict or instance of same class: expected <class 'google.protobuf.any_pb2.Any'> got ...whenever aMutateRowsRPC failed with rich gRPC error details._get_statusnow normalizes each item inexc.details, keepingany_pb2.Anymessages and packing unpackedgoogle.protobuf.message.Messageinstances intoany_pb2.Any.MutationsBatcherAsync/MutationsBatchererror handling:_execute_mutate_rows, only compute_get_statuses_from_mutations_exception_groupandStatuslists whenself._user_batch_completed_callback is not None(falling back toUNKNOWNstatuses if an unexpected non-MutationsExceptionGroupexception occurs).MutationsBatcherAsync._wait_for_batch_results, moveawait taskinside thetry:block so unexpected task exceptions in async mode are caught and wrapped inFailedMutationEntryErrorrather than aborting_flush_internalbefore_add_exceptions(which previously causedbatcher.close()to silently drop failed mutations)._get_error_typestatus code validation:core_exceptions.from_grpc_status(call_code, "")maps0(grpc.StatusCode.OK) and unmapped integers to the base classGoogleAPICallError, which causedretries.if_exception_typeto match and retry everyGoogleAPICallErrorsubclass._get_error_typenow raisesValueErrorfor0/grpc.StatusCode.OK, booleans, and any unmapped status code that falls back to baseGoogleAPICallError.Test plan
tests/unit/data/test__helpers.pyfor_get_statuspacking unpacked protobuf details and_get_error_typerejecting invalid status codes (0,grpc.StatusCode.OK, booleans, and unmapped codes).tests/unit/data/_async/test_mutations_batcher.py(and regeneratedtests/unit/data/_sync_autogen/test_mutations_batcher.py) for_execute_mutate_rowscallback fallback on unexpected errors and_wait_for_batch_resultscatching unexpected task exceptions.test_sync_up_to_date.pyand fullpackages/google-cloud-bigtable/tests/unitsuite pass.