zoo.pipeline.inference package

Submodules

zoo.pipeline.inference.inference_model module

class zoo.pipeline.inference.inference_model.InferenceModel(supported_concurrent_num=1, bigdl_type='float')[source]

Bases: bigdl.util.common.JavaValue

Model for thread-safe inference. To do inference, you need to first initiate an InferenceModel instance, then call load|load_caffe|load_openvino to load a pre-trained model, and finally call predict.

# Arguments supported_concurrent_num: Int. How many concurrent threads to invoke. Default is 1.

load(model_path, weight_path=None)[source]

Load a pre-trained Analytics Zoo or BigDL model.

Parameters:
  • model_path – String. The file path to the model.
  • weight_path – String. The file path to the weights if any. Default is None.
load_bigdl(model_path, weight_path=None)[source]

Load a pre-trained Analytics Zoo or BigDL model.

Parameters:
  • model_path – String. The file path to the model.
  • weight_path – String. The file path to the weights if any. Default is None.
load_caffe(model_path, weight_path)[source]

Load a pre-trained Caffe model.

Parameters:
  • model_path – String. The file path to the prototxt file.
  • weight_path – String. The file path to the Caffe model.
load_openvino(model_path, weight_path, batch_size=0)[source]

Load an OpenVINI IR.

Parameters:
  • model_path – String. The file path to the OpenVINO IR xml file.
  • weight_path – String. The file path to the OpenVINO IR bin file.
  • batch_size – Int. Set batch Size, default is 0 (use default batch size).
load_tensorflow(model_path, model_type, inputs, outputs, intra_op_parallelism_threads=1, inter_op_parallelism_threads=1, use_per_session_threads=True)[source]

Load an TensorFlow model using tensorflow.

Parameters:
  • model_path – String. The file path to the TensorFlow model.
  • model_type – String. The type of the tensorflow model file: “frozenModel” or “savedModel”.
  • inputs – Array[String]. the inputs of the model.

inputs outputs: Array[String]. the outputs of the model. :param intra_op_parallelism_threads: Int. The number of intraOpParallelismThreads.

Default is 1.
Parameters:
  • inter_op_parallelism_threads – Int. The number of interOpParallelismThreads. Default is 1.
  • use_per_session_threads – Boolean. Whether to use perSessionThreads. Default is True.
predict(inputs)[source]

Do prediction on inputs.

Parameters:inputs – A numpy array or a list of numpy arrays or JTensor or a list of JTensors.

Module contents