-
Notifications
You must be signed in to change notification settings - Fork 729
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
OpenVINO resnet_50_v1 python example with VNNI #1582
Changes from 9 commits
467c0cf
b5c7c25
e975ac6
782c75f
df55a76
beddf0d
ea85c9a
17eb177
b4681d1
4fa2682
e4e9cda
64293d7
e0d1986
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,52 @@ | ||
## OpenVINO ResNet_v1_50 example | ||
This example illustrates how to use a pre-trained OpenVINO optimized model to make inferences with OpenVINO toolkit as backend using Analytics Zoo. | ||
|
||
## Install or download Analytics Zoo | ||
Follow the instructions [here](https://analytics-zoo.github.io/master/#PythonUserGuide/install/) to install analytics-zoo via __pip__ or __download the prebuilt package__. | ||
|
||
## PrepareOpenVINOResNet | ||
TensorFlow models cannot be directly loaded by OpenVINO. It should be converted to OpenVINO optimized model and int8 optimized model first. You can use PrepareOpenVINOResNet or [OpenVINO toolkit](https://docs.openvinotoolkit.org/2018_R5/_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow.html) to finish this job. Herein, we focused on PrepareOpenVINOResNet. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. and -> or? |
||
|
||
Download [TensorFlow ResNet50_v1](http://download.tensorflow.org/models/resnet_v1_50_2016_08_28.tar.gz), [validation image set](https://s3-ap-southeast-1.amazonaws.com/analytics-zoo-models/openvino/val_bmp_32.tar) and [OpenCVLibs](https://s3-ap-southeast-1.amazonaws.com/analytics-zoo-models/openvino/opencv_4.0.0_ubuntu_lib.tar). Extract files from these packages. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we can just add a link to original document and remove model convert section. |
||
|
||
```bash | ||
export SPARK_HOME=the root directory of Spark | ||
export ANALYTICS_ZOO_HOME=the folder where you extract the downloaded Analytics Zoo zip package | ||
export ANALYTICS_ZOO_JAR=export ANALYTICS_ZOO_JAR=`find ${ANALYTICS_ZOO_HOME}/lib -type f -name "analytics-zoo*jar-with-dependencies.jar"` | ||
|
||
MODEL_PATH=dir of resetNet 50 checkpoint, i.e., resnet_v1_50.ckpt | ||
VALIDATION=dir of validation images and val.txt, i.e., val_bmp_32 | ||
OPENCVLIBS=dir of OpenCV libs | ||
|
||
java -cp ${ANALYTICS_ZOO_JAR}:${SPARK_HOME}/jars/* \ | ||
com.intel.analytics.zoo.examples.vnni.openvino.PrepareOpenVINOResNet \ | ||
-m ${MODEL_PATH} -v ${VALIDATION} -l ${OPENCVLIBS} | ||
``` | ||
|
||
__Options:__ | ||
- `-m` `--model`: The dir of resetNet 50 checkpoint. | ||
- `-b` `--batchSize`: The batch size of input data. Default is 4. | ||
- `-l` `--openCVLibs`: The number of iterations to run the performance test. Default is 1. | ||
- `-v` `--validationFilePath`: dir of validation images and val.txt. | ||
- `--subset`: Number of images in validation file path. Note that it should be align with val.txt. | ||
|
||
|
||
__Sample Result files in MODEL_PATH__: | ||
``` | ||
resnet_v1_50.ckpt | ||
resnet_v1_50_inference_graph.bin | ||
resnet_v1_50_inference_graph-calibrated.bin | ||
resnet_v1_50_inference_graph-calibrated.xml | ||
resnet_v1_50_inference_graph.mapping | ||
resnet_v1_50_inference_graph.xml | ||
``` | ||
|
||
Amount them, `resnet_v1_50_inference_graph.xml` and `resnet_v1_50_inference_graph.bin` are OpenVINO optimized ResNet_v1_50 model and weight, `resnet_v1_50_inference_graph-calibrated.xml` and `resnet_v1_50_inference_graph-calibrated.bin` are OpenVINO int8 optimized ResNet_v1_50 model and weight. Both of them can be loaded by OpenVINO or Zoo. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. amount or among? |
||
|
||
## Options | ||
* `--image` The path where the images are stored. It can be either a folder or an image path. Local file system, HDFS and Amazon S3 are supported. | ||
* `--model` The path to the TensorFlow object detection model. | ||
* `--partition_num` The number of partitions. | ||
|
||
## Results | ||
We print the inference result of each batch. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Pls add detailed running command for this example. :) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Any sample output to put here? |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,56 @@ | ||
from optparse import OptionParser | ||
|
||
from zoo.pipeline.inference import InferenceModel | ||
from zoo.common.nncontext import init_nncontext | ||
from zoo.feature.image import * | ||
from zoo.pipeline.nnframes import * | ||
|
||
batch_size = 4 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. global variable should in up case |
||
|
||
|
||
def predict(model_path, img_path, partition_num): | ||
model = InferenceModel() | ||
model.load_openvino(model_path, | ||
weight_path=model_path[:model_path.rindex(".")] + ".bin", | ||
batch_size=batch_size) | ||
sc = init_nncontext("OpenVINO Object Detection Inference Example") | ||
infer_transformer = ChainedPreprocessing([ImageBytesToMat(), | ||
ImageResize(256, 256), | ||
ImageCenterCrop(224, 224), | ||
ImageMatToTensor(format="NHWC", to_RGB=True)]) | ||
image_set = ImageSet.read(img_path, sc, partition_num).\ | ||
transform(infer_transformer).get_image().collect() | ||
image_set = np.expand_dims(image_set, axis=1) | ||
|
||
for i in range(len(image_set) // batch_size + 1): | ||
index = i * batch_size | ||
batch = image_set[index] | ||
for j in range(index + 1, min(index + batch_size, len(image_set))): | ||
batch = np.vstack((batch, image_set[j])) | ||
batch = np.expand_dims(batch, axis=0) | ||
|
||
predictions = model.predict(batch) | ||
|
||
result = predictions[0] | ||
|
||
print("batch_" + str(i)) | ||
for r in result: | ||
output = {} | ||
max_index = np.argmax(r) | ||
output["Top-1"] = str(max_index) | ||
print("* Predict result " + str(output)) | ||
|
||
|
||
if __name__ == "__main__": | ||
parser = OptionParser() | ||
parser.add_option("--image", type=str, dest="img_path", | ||
help="The path where the images are stored, " | ||
"can be either a folder or an image path") | ||
parser.add_option("--model", type=str, dest="model_path", | ||
help="Zoo Model Path") | ||
parser.add_option("--partition_num", type=int, dest="partition_num", default=4, | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think |
||
help="The number of partitions") | ||
|
||
(options, args) = parser.parse_args(sys.argv) | ||
|
||
predict(options.model_path, options.img_path, options.partition_num) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One blank line is enough