Skip to content

Feat: Zero-copy array.array parameters with atomic element access (#149) - #225

Merged
b-re-w merged 6 commits into
developfrom
feat/tp-buffer-arrays
Oct 17, 2026
Merged

b-re-w merged 6 commits into
developfrom
feat/tp-buffer-arrays

Conversation

@b-re-w

@b-re-w b-re-w commented Oct 17, 2026

Copy link
Copy Markdown
Member

Refs #149. The concurrent-resize behaviour is still awaiting a user decision, so the issue stays open. numpy is split out to #220.

Feature added:

  • array.array[float] and array.array[int] parameters compile without a copy. The entry guard checks the exact type, a C-contiguous 1-D buffer with format d or q/l, itemsize 8 and 8-byte alignment; anything else deopts before any effect.
  • The buffer export is held for the call and released on every exit (ok, error, deopt).
  • Every element access is one relaxed atomic 8-byte load or store, so each access is atomic like the interpreter's a[i]. There is no copy-in and no critical section, and no limit on the number of buffer parameters.
  • Two parameters that are the same array are correct: reads see earlier writes in program order.
  • Closedness (TypedPython: an ArrayParam function that calls user code compiles and diverges from CPython #203) applies.
  • IndexError text matches CPython (array index out of range).
  • -DTP_BUFFER_PLAIN switches element access to plain loads and stores, for measurement only.

Not done:

Tests added: test_buffer_arrays.py, 51 tests.

  • Single-thread differential against the interpreter: reads, writes, len, IndexError, negative indices, a wrong typecode taking the object path, aliasing, an exception mid-loop, an export released after every exit, closedness, and the plain switch.
  • Red before: 74 failed in the first version; the tests that passed then were vacuous until assert_compiled was added.
  • Mutation: dropping the release on the error path leaves append raising BufferError, so the test fails.

Measured (quiet window, 11 interleaved pairs, 2,000,000 doubles, 3.15.0rc1t; benchmarks/typedpython/results/2026-10-17-buffer-atomic-*):

kernel atomic / plain note
dot (read + reduction) 1.001 the float reduction is not vectorised anyway (strict FP)
scale (element-wise store) 2.539 the atomics block vectorisation of element-wise loops
dot, array.array vs list copy-in 1.94 ms vs 12.28 ms zero-copy is 6.3x faster

Verification (local, 3.15.0rc1t):

  • Full suite as CI runs it: 1138 passed, 3 skipped.
  • The compiled function sets of the five benchmarks equal develop's. They use lists, and nbody still needs the pending decision.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant