fix(mnn): use getNumpyData() instead of getData() for output extraction (#717)

MNNInferSession.__call__ extracted session output via
np.array(out_tensor.getData()).reshape(out_shape), which marshals
tensor contents through a slow per-element Python path. For a
PP-OCRv6 medium recognizer output (~17MB), this cost ~454ms versus
~210ms for the actual GPU inference that produced it.

Switch to getNumpyData(), which exposes the buffer directly. An
explicit copy is required since getNumpyData() returns a view into
memory owned by out_tensor; without it, out_tensor going out of
scope or being reused across shape changes caused intermittent
segfaults.

On a 39-box real-world image, total recognition time dropped from
~22s to ~8.4s. Per-crop output extraction dropped from ~454ms to
~10ms.

Fixes intermittent segfault risk from unmanaged zero-copy tensor
lifetime as a side effect.
This commit is contained in:
Horatiu George Todoran authored and GitHub committed 2026-08-18 08:15:26 +08:00
1 parent 3efd66a6ba
commit 667d04c029
1 file changed
+1 -2
+1 -2
View File
@@ -73,8 +73,7 @@ class MNNInferSession(InferSession):
out_shape, MNN.Halide_Type_Float, MNN.Tensor_DimensionType_Caffe
)
output.copyToHostTensor(out_tensor)
return np.array(out_tensor.getData()).reshape(out_shape)
return np.array(out_tensor.getNumpyData(), copy=True).reshape(out_shape)
except Exception as e:
error_info = traceback.format_exc()