* feat(inference_engine): add TensorRT engine support
- Add TensorRT inference session with dynamic shape support
- Implement automatic engine building and caching per GPU architecture
- Add pre-allocated buffers with pinned memory optimization
- Support FP16/FP32/INT8 precision modes
- Add optimization profiles for det/rec/cls models
- Include comprehensive error handling and resource cleanup
- Add unit tests for TensorRT engine (skipped when unavailable)
- Add TensorRT configuration section in config.yaml
* fix(tensorrt): dynamically allocate output buffers to prevent overflow
* Update test_engine.py with TensorRT support final version
* feat(tensorrt): add model_type to engine cache key
- Include model_type in TensorRT engine cache key generation
- Ensures proper cache differentiation based on model_type
* fix(tensorrt): update maximum shape configurations for input and output tensors
- Increased max_shape values in config.yaml and engine_builder.py to 2048 for both detection and recognition tasks.
- Updated default maximum shapes in memory_utils.py to reflect the new limits for input and output buffers.
* feat(tensorrt): implement square input handling for MULTI models
- Added functionality to check if the model requires square input based on configuration.
- Implemented padding of input to square shape and cropping of output back to original dimensions.
- Introduced methods to determine maximum profile size and handle square input requirements for TensorRT inference.
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
locviettriandCursor
SWHL