近几年,随着“新零售”概念的火热,越来越多的品牌商及商超零售企业开始引进人工智能技术,探索商品管理、成本控制、用户体验等多维度的数字化转型,转型中所涉及的货架陈列分析、智能结算、智能库存管理、智能货柜、电商平台等以图搜图的场景背后的核心技术都离不开商品识别算法。
☞☞☞AI 智能聊天, 问答助手, AI 智能搜索, 免费无限量使用 DeepSeek R1 模型☜☜☜


痛点问题
解决方案
PaddleClas团队开源的图像识别PP-ShiTu技术方案,主要由主体检测、特征学习和向量检索三个模块组成,是一个实用的轻量级通用图像识别系统。基于此技术方案,商超零售企业可实现大量商品的一键式智能化识别,大大提高识别效率,节省人工及时间成本。
此外,当新品迭代更新时,PP-shitu无需重新训练模型,能够做到“即增即用”,完美解决上述痛点问题,大大提高了人工智能在商超零售行业的应用落地可能性。
PP-shitu技术方案可具体应用于例如:商品结算、库存管理等关于商品识别的商超细分场景。
模型工具简介
飞桨图像识别套件PaddleClas是飞桨为工业界和学术界所准备的一个图像识别任务的工具集,助力使用者训练出更好的视觉模型和应用落地。
而PP-ShiTu是一个实用的轻量级通用图像识别系统,主要由主体检测、特征学习和向量检索三个模块组成。该系统从骨干网络选择和调整、损失函数的选择、数据增强、学习率变换策略、正则化参数选择、预训练模型使用以及模型裁剪量化8个方面,采用多种策略,对各个模块的模型进行优化,最终得到在CPU上仅0.2s即可完成10w+库的图像识别的系统。
环境要求
Python >= 3.6
PaddlePaddle >= 2.1
Linux 环境最佳
安装PaddleClas
# 克隆 PaddleClas# 由于本项目已克隆并存储,故不重复克隆## github仓库#!git clone https://github.com/PaddlePaddle/PaddleClas.git -b release/2.3 ## gitee仓库(推荐)#!git clone https://gitee.com/paddlepaddle/PaddleClas.git -b release/2.3
# 安装 Python 依赖库 # PaddleClas 的 Python 依赖库在 `requirements.txt` 中给出,可通过如下命令安装:%cd /home/aistudio/PaddleClas/ !pip install --upgrade -r requirements.txt -i https://mirror.baidu.com/pypi/simple
数据集介绍
商品部分
东古酱油一品鲜
东古黄豆酱750G
东鹏特饮罐装
中华(硬)
中华(软)
ru酸菌600亿_2
ru酸菌600亿_3
ru酸菌600亿原味
ru酸菌600亿芒果
ru酸菌600亿芦荟
...
数据集格式
# 采用"空格"作为分隔符号... train/10/1283.jpg 10 624 train/10/1284.jpg 10 625 train/10/1285.jpg 10 626 train/10/1286.jpg 10 627 ...
验证数据集(本数据集中既是 gallery dataset,也是 query dataset)test_list.txt 文件内容格式如下所示:
... test/103/743.jpg 103 743 test/103/744.jpg 103 744 test/103/745.jpg 103 745 test/103/746.jpg 103 746 ...
注:
├── classlabel.txt # 数据label和对应商品名称├── gallery_label.txt # 图片地址及对应的商品名称├── gallery # 底库图片├── test # 测试集图片├── test_list.txt # 测试集图片地址及对应的商品名称├── train # 训练集图片└── train_list.txt # 训练集图片地址及对应的商品名称
# 只想体验项目部署的,可解压此数据集,直接进入【8. 图像识别系统】%cd /home/aistudio/# !unzip -qd dataset dataset.zip## 想体验模型训练及评估等全流程的,解压data目录下数据集文件,本数据集较大,第一次下载需要耐心等待~!unzip -qd dataset /home/aistudio/data/data108651/dataset.zip
PP-ShiTu是一个实用的轻量级通用图像识别系统,主要由主体检测、特征学习和向量检索三个模块组成。该系统从骨干网络选择和调整、损失函数的选择、数据增强、学习率变换策略、正则化参数选择、预训练模型使用以及模型裁剪量化8个方面,采用多种策略,对各个模块的模型进行优化,最终得到在CPU上仅0.2s即可完成10w+库的图像识别的系统。
主体检测
主体检测技术是目前应用非常广泛的一种检测技术,它指的是检测出图片中一个或者多个主体的坐标位置,然后将图像中的对应区域裁剪下来,进行识别,从而完成整个识别过程。主体检测是识别任务的前序步骤,可以有效提升识别精度。
考虑到商品识别实际应用场景中,需要快速准确地获得识别结果,故本项目选取适用于 CPU 或者移动端场景的轻量级主体检测模型PicoDet作为本项目主体检测部分的模型。此模型融合了ATSS、Generalized Focal Loss、余弦学习率策略、Cycle-EMA、轻量级检测 head等一系列优化算法,基于COCO train2017数据集进行大规模预训练,最终inference模型大小(MB)仅30.1MB,mAP可达40.1%,在cpu下单张图片预测耗时仅29.8ms,完美符合本项目实际落地需求,故在本项目中不对主体检测部分做适应性训练。
特征提取
特征提取是图像识别中的关键一环,它的作用是将输入的图片转化为固定维度的特征向量,用于后续的向量检索。好的特征需要具备相似度保持性,即在特征空间中,相似度高的图片对其特征相似度要比较高(距离比较近),相似度低的图片对,其特征相似度要比较小(距离比较远)。Deep Metric Learning用以研究如何通过深度学习的方法获得具有强表征能力的特征。
考虑到本项目的真实落地的场景中,推理速度及预测准确率是考量模型好坏的重要指标,所以本项目采用 PP_LCNet_x2_5 作为骨干网络, Neck 部分选用 Linear Layer, Head 部分选用 ArcMargin,Loss 部分选用 CELoss,并结合度量学习arcmargin算法,对高相似物体的区分效果远超单一模型,能更好地适应 Intel CPU,不仅准确率超越大模型ResNet50,预测速度还能快上3倍。
向量检索
向量检索技术在图像识别、图像检索中应用比较广泛。其主要目标是,对于给定的查询向量,在已经建立好的向量库中,与库中所有的待查询向量,进行特征向量的相似度或距离计算,得到相似度排序。在图像识别系统中,本项目使用 Faiss 对此部分进行支持。在此过程中,本项目选取 HNSW32 为检索算法,使得检索精度、检索速度能够取得较好的平衡,更为贴切本项目实际应用场景的使用需求。
本项目在主体检测部分直接采用PaddleDetection提供给得预训练模型,故这里主要介绍特征提取部分的模型训练。
修改配置文件
首先,在启动模型训练之前,需要在配置文件中修改数据配置相关的内容, 主要包括数据集的地址以及类别数量。对应到配置文件中的位置如下所示:
Head:
name: ArcMargin
embedding_size: 512
class_num: 358 #此处表示类别数Train: dataset: name: ImageNetDataset image_root: /home/aistudio/dataset/ #此处表示train数据所在的目录 cls_label_path: /home/aistudio/dataset/train_list.txt #此处表示train数据集label文件的地址
Query: dataset: name: VeriWild image_root: /home/aistudio/dataset/ #此处表示query数据集所在的目录 cls_label_path: /home/aistudio/dataset/test_list.txt #此处表示query数据集label文件的地址
Gallery: dataset: name: VeriWild image_root: /home/aistudio/dataset/ #此处表示gallery数据集所在的目录 cls_label_path: /home/aistudio/dataset/test_list.txt #此处表示gallery数据集label文件的地址
模型训练
%cd /home/aistudio/PaddleClas
!python tools/train.py \
-c ./ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml \
-o Arch.Backbone.pretrained=True \
-o Global.device=gpu \%cd /home/aistudio/PaddleClas
!python tools/eval.py \
-c ./ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml \
-o Global.pretrained_model="/home/aistudio/PaddleClas/output/RecModel/best_model"由评估log可见,recall1为0.98232,能够符合实际产业场景应用需求。
%cd /home/aistudio/PaddleClas
!python tools/export_model.py \
-c ./ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml \
-o Global.pretrained_model="output/RecModel/best_model"生成的推理模型位于 inference 目录,里面包含三个文件,分别为 inference.pdmodel、inference.pdiparams、inference.pdiparams.info。 其中: inference.pdmodel 用来存储推理模型的结构, inference.pdiparams 和 inference.pdiparams.info 用来存储推理模型相关的参数信息。
获取特征向量
%cd /home/aistudio/PaddleClas/deploy
!python python/predict_rec.py \
-c configs/inference_rec.yaml \
-o Global.rec_inference_model_dir="../inference" \
-o Global.infer_imgs="/home/aistudio/dataset/test_image/sijibao.jpg"这里串联主体检测、特征提取、向量检索,从而构成一整套图像识别系统:
Global: rec_inference_model_dir: "/home/aistudio/PaddleClas/inference"
IndexProcess: index_method: "HNSW32" # supported: HNSW32, IVF, Flat image_root: "/home/aistudio/dataset/" index_dir: "/home/aistudio/dataset/index" data_file: "/home/aistudio/dataset/gallery_label.txt"
执行如下代码:
# 建立索引库%cd /home/aistudio/PaddleClas/deploy
!python3 python/build_gallery.py \
-c configs/build_general.yaml \
-o IndexProcess.data_file="/home/aistudio/dataset/gallery_label.txt" \
-o IndexProcess.index_dir="/home/aistudio/dataset/index"%cd /home/aistudio/PaddleClas/deploy/ %mkdir models %cd models# 下载通用检测 inference 模型并解压!wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar && tar -xf picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar
修改推理文件configs/inference_general.yaml内容:
Global: infer_imgs: "/home/aistudio/dataset/test_image/sijibao.jpg" det_inference_model_dir: "./models/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer" rec_inference_model_dir: "/home/aistudio/PaddleClas/inference" rec_nms_thresold: 0.05
IndexProcess: index_dir: "/home/aistudio/dataset/index" return_k: 5 score_thres: 0.5
运行下面的命令,对图像 /home/aistudio/dataset/test_image/sijibao.jpg 进行识别与检索:
#基于索引库的图像识别%cd /home/aistudio/PaddleClas/deploy
!python python/predict_system.py \
-c configs/inference_general.yaml \
-o Global.infer_imgs="/home/aistudio/dataset/test_image/sijibao.jpg" \
-o IndexProcess.index_dir="/home/aistudio/dataset/index"/home/aistudio/PaddleClas/deploy
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/vision/transforms/functional_pil.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': Image.NEAREST,
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/vision/transforms/functional_pil.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': Image.BILINEAR,
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/vision/transforms/functional_pil.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': Image.BICUBIC,
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/vision/transforms/functional_pil.py:39: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': Image.BOX,
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/vision/transforms/functional_pil.py:40: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': Image.LANCZOS,
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/vision/transforms/functional_pil.py:41: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': Image.HAMMING
2022-04-07 16:40:27 INFO:
===========================================================
== PaddleClas is powered by PaddlePaddle ! ==
===========================================================
== ==
== For more info please go to the following website. ==
== ==
== https://github.com/PaddlePaddle/PaddleClas ==
===========================================================
2022-04-07 16:40:27 INFO: DetPostProcess :
2022-04-07 16:40:27 INFO: DetPreProcess :
2022-04-07 16:40:27 INFO: transform_ops :
2022-04-07 16:40:27 INFO: DetResize :
2022-04-07 16:40:27 INFO: interp : 2
2022-04-07 16:40:27 INFO: keep_ratio : False
2022-04-07 16:40:27 INFO: target_size : [640, 640]
2022-04-07 16:40:27 INFO: DetNormalizeImage :
2022-04-07 16:40:27 INFO: is_scale : True
2022-04-07 16:40:27 INFO: mean : [0.485, 0.456, 0.406]
2022-04-07 16:40:27 INFO: std : [0.229, 0.224, 0.225]
2022-04-07 16:40:27 INFO: DetPermute :
2022-04-07 16:40:27 INFO: Global :
2022-04-07 16:40:27 INFO: batch_size : 1
2022-04-07 16:40:27 INFO: cpu_num_threads : 10
2022-04-07 16:40:27 INFO: det_inference_model_dir : ./models/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer
2022-04-07 16:40:27 INFO: enable_benchmark : True
2022-04-07 16:40:27 INFO: enable_mkldnn : True
2022-04-07 16:40:27 INFO: enable_profile : False
2022-04-07 16:40:27 INFO: gpu_mem : 8000
2022-04-07 16:40:27 INFO: image_shape : [3, 640, 640]
2022-04-07 16:40:27 INFO: infer_imgs : /home/aistudio/dataset/test_image/sijibao.jpg
2022-04-07 16:40:27 INFO: ir_optim : True
2022-04-07 16:40:27 INFO: labe_list : ['foreground']
2022-04-07 16:40:27 INFO: max_det_results : 5
2022-04-07 16:40:27 INFO: rec_inference_model_dir : /home/aistudio/PaddleClas/inference
2022-04-07 16:40:27 INFO: rec_nms_thresold : 0.05
2022-04-07 16:40:27 INFO: threshold : 0.2
2022-04-07 16:40:27 INFO: use_fp16 : False
2022-04-07 16:40:27 INFO: use_gpu : True
2022-04-07 16:40:27 INFO: use_tensorrt : False
2022-04-07 16:40:27 INFO: IndexProcess :
2022-04-07 16:40:27 INFO: batch_size : 32
2022-04-07 16:40:27 INFO: data_file : /home/aistudio/dataset/gallery_label.txt
2022-04-07 16:40:27 INFO: delimiter :
2022-04-07 16:40:27 INFO: dist_type : IP
2022-04-07 16:40:27 INFO: embedding_size : 512
2022-04-07 16:40:27 INFO: image_root : /home/aistudio/dataset/
2022-04-07 16:40:27 INFO: index_dir : /home/aistudio/dataset/index
2022-04-07 16:40:27 INFO: index_method : HNSW32
2022-04-07 16:40:27 INFO: index_operation : new
2022-04-07 16:40:27 INFO: return_k : 5
2022-04-07 16:40:27 INFO: score_thres : 0.5
2022-04-07 16:40:27 INFO: RecPostProcess : None
2022-04-07 16:40:27 INFO: RecPreProcess :
2022-04-07 16:40:27 INFO: transform_ops :
2022-04-07 16:40:27 INFO: ResizeImage :
2022-04-07 16:40:27 INFO: size : 224
2022-04-07 16:40:27 INFO: NormalizeImage :
2022-04-07 16:40:27 INFO: mean : [0.485, 0.456, 0.406]
2022-04-07 16:40:27 INFO: order :
2022-04-07 16:40:27 INFO: scale : 0.00392157
2022-04-07 16:40:27 INFO: std : [0.229, 0.224, 0.225]
2022-04-07 16:40:27 INFO: ToCHWImage : None
/home/aistudio/PaddleClas/deploy/python/preprocess.py:65: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': Image.NEAREST,
/home/aistudio/PaddleClas/deploy/python/preprocess.py:66: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': Image.BILINEAR,
/home/aistudio/PaddleClas/deploy/python/preprocess.py:67: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': Image.BICUBIC,
/home/aistudio/PaddleClas/deploy/python/preprocess.py:68: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': Image.BOX,
/home/aistudio/PaddleClas/deploy/python/preprocess.py:69: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': Image.LANCZOS,
/home/aistudio/PaddleClas/deploy/python/preprocess.py:70: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': Image.HAMMING
Inference: 34.667015075683594 ms per batch image
[{'bbox': [131, 114, 356, 355], 'rec_docs': '四季宝花生酱', 'rec_scores': 0.85414684}]其中 bbox 表示检测出的主体所在位置,rec_docs 表示索引库中与检测框最为相似的类别,rec_scores 表示对应的置信度。
检测的可视化结果也保存在 output 文件夹下,对于本张图像,识别结果可视化如下所示:


运行如下识别命令:
%cd /home/aistudio/PaddleClas/deploy
!python python/predict_system.py \
-c configs/inference_general.yaml \
-o Global.infer_imgs="/home/aistudio/dataset/test_image/recognition_2.jpg"/home/aistudio/PaddleClas/deploy /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/vision/transforms/functional_pil.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead. 'nearest': Image.NEAREST, /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/vision/transforms/functional_pil.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead. 'bilinear': Image.BILINEAR, /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/vision/transforms/functional_pil.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead. 'bicubic': Image.BICUBIC, /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/vision/transforms/functional_pil.py:39: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead. 'box': Image.BOX, /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/vision/transforms/functional_pil.py:40: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead. 'lanczos': Image.LANCZOS, /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/vision/transforms/functional_pil.py:41: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead. 'hamming': Image.HAMMING 2022-04-07 16:41:11 INFO: =========================================================== == PaddleClas is powered by PaddlePaddle ! == =========================================================== == == == For more info please go to the following website. == == == == https://github.com/PaddlePaddle/PaddleClas == =========================================================== 2022-04-07 16:41:11 INFO: DetPostProcess : 2022-04-07 16:41:11 INFO: DetPreProcess : 2022-04-07 16:41:11 INFO: transform_ops : 2022-04-07 16:41:11 INFO: DetResize : 2022-04-07 16:41:11 INFO: interp : 2 2022-04-07 16:41:11 INFO: keep_ratio : False 2022-04-07 16:41:11 INFO: target_size : [640, 640] 2022-04-07 16:41:11 INFO: DetNormalizeImage : 2022-04-07 16:41:11 INFO: is_scale : True 2022-04-07 16:41:11 INFO: mean : [0.485, 0.456, 0.406] 2022-04-07 16:41:11 INFO: std : [0.229, 0.224, 0.225] 2022-04-07 16:41:11 INFO: DetPermute : 2022-04-07 16:41:11 INFO: Global : 2022-04-07 16:41:11 INFO: batch_size : 1 2022-04-07 16:41:11 INFO: cpu_num_threads : 10 2022-04-07 16:41:11 INFO: det_inference_model_dir : ./models/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer 2022-04-07 16:41:11 INFO: enable_benchmark : True 2022-04-07 16:41:11 INFO: enable_mkldnn : True 2022-04-07 16:41:11 INFO: enable_profile : False 2022-04-07 16:41:11 INFO: gpu_mem : 8000 2022-04-07 16:41:11 INFO: image_shape : [3, 640, 640] 2022-04-07 16:41:11 INFO: infer_imgs : /home/aistudio/dataset/test_image/recognition_2.jpg 2022-04-07 16:41:11 INFO: ir_optim : True 2022-04-07 16:41:11 INFO: labe_list : ['foreground'] 2022-04-07 16:41:11 INFO: max_det_results : 5 2022-04-07 16:41:11 INFO: rec_inference_model_dir : /home/aistudio/PaddleClas/inference 2022-04-07 16:41:11 INFO: rec_nms_thresold : 0.05 2022-04-07 16:41:11 INFO: threshold : 0.2 2022-04-07 16:41:11 INFO: use_fp16 : False 2022-04-07 16:41:11 INFO: use_gpu : True 2022-04-07 16:41:11 INFO: use_tensorrt : False 2022-04-07 16:41:11 INFO: IndexProcess : 2022-04-07 16:41:11 INFO: batch_size : 32 2022-04-07 16:41:11 INFO: data_file : /home/aistudio/dataset/gallery_label.txt 2022-04-07 16:41:11 INFO: delimiter : 2022-04-07 16:41:11 INFO: dist_type : IP 2022-04-07 16:41:11 INFO: embedding_size : 512 2022-04-07 16:41:11 INFO: image_root : /home/aistudio/dataset/ 2022-04-07 16:41:11 INFO: index_dir : /home/aistudio/dataset/index 2022-04-07 16:41:11 INFO: index_method : HNSW32 2022-04-07 16:41:11 INFO: index_operation : new 2022-04-07 16:41:11 INFO: return_k : 5 2022-04-07 16:41:11 INFO: score_thres : 0.5 2022-04-07 16:41:11 INFO: RecPostProcess : None 2022-04-07 16:41:11 INFO: RecPreProcess : 2022-04-07 16:41:11 INFO: transform_ops : 2022-04-07 16:41:11 INFO: ResizeImage : 2022-04-07 16:41:11 INFO: size : 224 2022-04-07 16:41:11 INFO: NormalizeImage : 2022-04-07 16:41:11 INFO: mean : [0.485, 0.456, 0.406] 2022-04-07 16:41:11 INFO: order : 2022-04-07 16:41:11 INFO: scale : 0.00392157 2022-04-07 16:41:11 INFO: std : [0.229, 0.224, 0.225] 2022-04-07 16:41:11 INFO: ToCHWImage : None /home/aistudio/PaddleClas/deploy/python/preprocess.py:65: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead. 'nearest': Image.NEAREST, /home/aistudio/PaddleClas/deploy/python/preprocess.py:66: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead. 'bilinear': Image.BILINEAR, /home/aistudio/PaddleClas/deploy/python/preprocess.py:67: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead. 'bicubic': Image.BICUBIC, /home/aistudio/PaddleClas/deploy/python/preprocess.py:68: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead. 'box': Image.BOX, /home/aistudio/PaddleClas/deploy/python/preprocess.py:69: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead. 'lanczos': Image.LANCZOS, /home/aistudio/PaddleClas/deploy/python/preprocess.py:70: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead. 'hamming': Image.HAMMING Inference: 34.21902656555176 ms per batch image []
输出结果为空。
由于默认的索引库中不包含对应的索引信息,所以这里的识别结果有误,此时我们可以通过构建新的索引库的方式,完成未知类别的图像识别。
当索引库中的图像无法覆盖我们实际识别的场景时,即在预测未知类别的图像时,只需要将对应类别的相似图像添加到索引库中,从而完成对未知类别的图像识别,这一过程是不需要重新训练的。
准备新的数据与标签
首先需要将与待检索图像相似的图像列表拷贝到索引库原始图像的文件夹。这里将所有的底库图像数据都放在文件夹 /home/aistudio/dataset/gallery/ 中。
然后需要编辑记录了图像路径和标签信息的文本文件,这里 PaddleClas 将更正后的标签信息文件放在了 /home/aistudio/dataset/gallery_update.txt 文件中。可以与原来的 /home/aistudio/dataset/gallery_label.txt 标签文件进行对比,添加了小度充电宝和韩国进口火山泥的索引图像。
每一行的文本中,第一个字段表示图像的相对路径,第二个字段表示图像对应的标签信息,中间用 \t 键分隔开
%cd /home/aistudio/PaddleClas/deploy/
!python python/build_gallery.py \
-c configs/build_general.yaml \
-o IndexProcess.data_file="/home/aistudio/dataset/gallery_update.txt" \
-o IndexProcess.index_dir="/home/aistudio/dataset/index_update"最终新的索引信息保存在文件夹 /home/aistudio/dataset/index_update 中。
使用新的索引库,对上述图像进行识别,运行命令如下:
%cd /home/aistudio/PaddleClas/deploy/
!python python/predict_system.py \
-c configs/inference_general.yaml \
-o Global.infer_imgs="/home/aistudio/dataset/test_image/recognition_2.jpg" \
-o IndexProcess.index_dir="/home/aistudio/dataset/index_update"/home/aistudio/PaddleClas/deploy
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/vision/transforms/functional_pil.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': Image.NEAREST,
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/vision/transforms/functional_pil.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': Image.BILINEAR,
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/vision/transforms/functional_pil.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': Image.BICUBIC,
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/vision/transforms/functional_pil.py:39: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': Image.BOX,
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/vision/transforms/functional_pil.py:40: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': Image.LANCZOS,
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/vision/transforms/functional_pil.py:41: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': Image.HAMMING
2022-04-07 16:42:20 INFO:
===========================================================
== PaddleClas is powered by PaddlePaddle ! ==
===========================================================
== ==
== For more info please go to the following website. ==
== ==
== https://github.com/PaddlePaddle/PaddleClas ==
===========================================================
2022-04-07 16:42:20 INFO: DetPostProcess :
2022-04-07 16:42:20 INFO: DetPreProcess :
2022-04-07 16:42:20 INFO: transform_ops :
2022-04-07 16:42:20 INFO: DetResize :
2022-04-07 16:42:20 INFO: interp : 2
2022-04-07 16:42:20 INFO: keep_ratio : False
2022-04-07 16:42:20 INFO: target_size : [640, 640]
2022-04-07 16:42:20 INFO: DetNormalizeImage :
2022-04-07 16:42:20 INFO: is_scale : True
2022-04-07 16:42:20 INFO: mean : [0.485, 0.456, 0.406]
2022-04-07 16:42:20 INFO: std : [0.229, 0.224, 0.225]
2022-04-07 16:42:20 INFO: DetPermute :
2022-04-07 16:42:20 INFO: Global :
2022-04-07 16:42:20 INFO: batch_size : 1
2022-04-07 16:42:20 INFO: cpu_num_threads : 10
2022-04-07 16:42:20 INFO: det_inference_model_dir : ./models/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer
2022-04-07 16:42:20 INFO: enable_benchmark : True
2022-04-07 16:42:20 INFO: enable_mkldnn : True
2022-04-07 16:42:20 INFO: enable_profile : False
2022-04-07 16:42:20 INFO: gpu_mem : 8000
2022-04-07 16:42:20 INFO: image_shape : [3, 640, 640]
2022-04-07 16:42:20 INFO: infer_imgs : /home/aistudio/dataset/test_image/recognition_2.jpg
2022-04-07 16:42:20 INFO: ir_optim : True
2022-04-07 16:42:20 INFO: labe_list : ['foreground']
2022-04-07 16:42:20 INFO: max_det_results : 5
2022-04-07 16:42:20 INFO: rec_inference_model_dir : /home/aistudio/PaddleClas/inference
2022-04-07 16:42:20 INFO: rec_nms_thresold : 0.05
2022-04-07 16:42:20 INFO: threshold : 0.2
2022-04-07 16:42:20 INFO: use_fp16 : False
2022-04-07 16:42:20 INFO: use_gpu : True
2022-04-07 16:42:20 INFO: use_tensorrt : False
2022-04-07 16:42:20 INFO: IndexProcess :
2022-04-07 16:42:20 INFO: batch_size : 32
2022-04-07 16:42:20 INFO: data_file : /home/aistudio/dataset/gallery_label.txt
2022-04-07 16:42:20 INFO: delimiter :
2022-04-07 16:42:20 INFO: dist_type : IP
2022-04-07 16:42:20 INFO: embedding_size : 512
2022-04-07 16:42:20 INFO: image_root : /home/aistudio/dataset/
2022-04-07 16:42:20 INFO: index_dir : /home/aistudio/dataset/index_update
2022-04-07 16:42:20 INFO: index_method : HNSW32
2022-04-07 16:42:20 INFO: index_operation : new
2022-04-07 16:42:20 INFO: return_k : 5
2022-04-07 16:42:20 INFO: score_thres : 0.5
2022-04-07 16:42:20 INFO: RecPostProcess : None
2022-04-07 16:42:20 INFO: RecPreProcess :
2022-04-07 16:42:20 INFO: transform_ops :
2022-04-07 16:42:20 INFO: ResizeImage :
2022-04-07 16:42:20 INFO: size : 224
2022-04-07 16:42:20 INFO: NormalizeImage :
2022-04-07 16:42:20 INFO: mean : [0.485, 0.456, 0.406]
2022-04-07 16:42:20 INFO: order :
2022-04-07 16:42:20 INFO: scale : 0.00392157
2022-04-07 16:42:20 INFO: std : [0.229, 0.224, 0.225]
2022-04-07 16:42:20 INFO: ToCHWImage : None
/home/aistudio/PaddleClas/deploy/python/preprocess.py:65: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': Image.NEAREST,
/home/aistudio/PaddleClas/deploy/python/preprocess.py:66: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': Image.BILINEAR,
/home/aistudio/PaddleClas/deploy/python/preprocess.py:67: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': Image.BICUBIC,
/home/aistudio/PaddleClas/deploy/python/preprocess.py:68: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
'box': Image.BOX,
/home/aistudio/PaddleClas/deploy/python/preprocess.py:69: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': Image.LANCZOS,
/home/aistudio/PaddleClas/deploy/python/preprocess.py:70: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': Image.HAMMING
Inference: 36.23652458190918 ms per batch image
[{'bbox': [2152, 179, 2916, 1709], 'rec_docs': '韩国进口火山泥', 'rec_scores': 0.5329636}, {'bbox': [572, 361, 1245, 1653], 'rec_docs': '小度充电宝', 'rec_scores': 0.5052846}]由测试效果图可知,模型对于未参与训练的商品及多个商品均有较好的识别效果:

PP-ShiTu中检测模型采用的 PicoDet算法,在使用官方模型后,如果不满足精度需求,则可以参考此部分文档,进行模型调优
对模型进行训练的话,需要自行准备数据,并对数据进行标注,建议一个类别至少准备200张标注图像,并将标注图像及groudtruth文件转成coco文件格式,以方便使用PaddleDetection进行训练的时候,请加载主体检测的预训练权重。
在使用官方模型后,如果不满足精度需求,则可以参考此部分文档,进行模型调优
因为要对模型进行训练,所以参照数据准备部分描述收集自己的数据集。值得注意的是,此部分需要准备大量的数据,以保证识别模型效果。
模型训练完成后,参照测试代码进行检索库更新。同时,对整个pipeline进行测试,如果精度不达预期,则重复此步骤。
使用 PaddleServing 做服务化部署时,需要将保存的 inference 模型转换为 Serving 模型。
模型转换
%cd /home/aistudio/PaddleClas/deploy/# 安装相关包!pip3 install paddle-serving-client==0.7.0 -i https://mirror.baidu.com/pypi/simple !pip3 install paddle-serving-app==0.7.0 -i https://mirror.baidu.com/pypi/simple !pip3 install faiss-cpu==1.7.1post2 -i https://mirror.baidu.com/pypi/simple !pip3 install paddle-serving-server-gpu==0.7.0.post102 -i https://mirror.baidu.com/pypi/simple !pip3 install paddle-serving-server==0.7.0 # CPU
# 执行模型转换命令!python3 -m paddle_serving_client.convert --dirname /home/aistudio/PaddleClas/inference/ \
--model_filename inference.pdmodel \
--params_filename inference.pdiparams \
--serving_server ./inference_PPLCNet_serving/ \
--serving_client ./inference_PPLCNet_client识别推理模型转换完成后,会在当前文件夹多出 inference_PPLCNet_serving/ 和 inference_PPLCNet_client/ 的文件夹。分别修改 inference_PPLCNet_serving/ 和 inference_PPLCNet_client/ 目录下的 serving_server_conf.prototxt 中的 alias 名字: 将 fetch_var 中的 alias_name 改为 features。 修改后的 serving_server_conf.prototxt 内容如下:
feed_var { name: "x" alias_name: "x" is_lod_tensor: false feed_type: 1 shape: 3 shape: 224 shape: 224}fetch_var { name: "save_infer_model/scale_0.tmp_1" alias_name: "features" is_lod_tensor: false fetch_type: 1 shape: 512}# 转换通用检测模型%cd /home/aistudio/PaddleClas/deploy/
!python -m paddle_serving_client.convert --dirname ./models/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer/ \
--model_filename inference.pdmodel \
--params_filename inference.pdiparams \
--serving_server ./picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/ \
--serving_client ./picodet_PPLCNet_x2_5_mainbody_lite_v1.0_client/检测 inference 模型转换完成后,会在当前文件夹多出 picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/ 和 picodet_PPLCNet_x2_5_mainbody_lite_v1.0_client/ 的文件夹。
注意: 此处不需要修改 picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/ 目录下的 serving_server_conf.prototxt 中的 alias 名字。
服务部署和请求
注意: 识别服务涉及到多个模型,出于性能考虑采用 PipeLine 部署方式。
%cd /home/aistudio/PaddleClas/deploy/paddleserving/recognition
/home/aistudio/PaddleClas/deploy/paddleserving/recognition
paddleserving 目录包含启动 pipeline 服务和发送预测请求的代码,包括:
__init__.pyconfig.yml # 启动服务的配置文件pipeline_http_client.py # http方式发送pipeline预测请求的脚本pipeline_rpc_client.py # rpc方式发送pipeline预测请求的脚本recognition_web_service.py # 启动pipeline服务端的脚本
PaddleServing Python Pipeline提供了用户友好的多模型组合服务编程框架,适用于多模型组合应用的场景。 其配置文件为YAML格式,一般默认为config.yaml。所以在启动服务前需要修改config.yaml配置文件。 这里主要修改识别模型和检测模型的路径以及相关的计算设备:
op: rec:
··· #uci模型路径 model_config: /home/aistudio/PaddleClas/deploy/inference_PPLCNet_serving
#计算硬件类型: 空缺时由devices决定(CPU/GPU),0=cpu, 1=gpu, 2=tensorRT, 3=arm cpu, 4=kunlun xpu device_type: 1
#计算硬件ID,当devices为""或不写时为CPU预测;当devices为"0", "0,1,2"时为GPU预测,表示使用的GPU卡 devices: "0" # "0,1"
#client类型,包括brpc, grpc和local_predictor.local_predictor不启动Serving服务,进程内预测 client_type: local_predictor #Fetch结果列表,以client_config中fetch_var的alias_name为准 fetch_list: ["features"] det: concurrency: 1 local_service_conf: client_type: local_predictor device_type: 1 devices: '0' fetch_list:
- save_infer_model/scale_0.tmp_1 model_config: /home/aistudio/PaddleClas/deploy/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving ···
index_dir = "/home/aistudio/dataset/index_update"
assert os.path.exists(os.path.join(
index_dir, "vector.index")), "vector.index not found ..."
assert os.path.exists(os.path.join(
index_dir, "id_map.pkl")), "id_map.pkl not found ... "
···# 启动服务,运行日志保存在 log.txt# 需要移步至终端执行此命令!python3 recognition_web_service.py &>log.txt &
!python3 pipeline_http_client.py
{'err_no': 0, 'err_msg': '', 'key': ['result'], 'value': ["[{'bbox': [132, 118, 355, 357], 'rec_docs': '四季宝花生酱', 'rec_scores': 0.85405}]"], 'tensors': []}
Used Time: 0.16031885147094727s# 此步需要移步至终端执行!python3 -m paddle_serving_server.serve stop
以上就是基于图像识别的智慧零售商品识别的详细内容,更多请关注php中文网其它相关文章!
每个人都需要一台速度更快、更稳定的 PC。随着时间的推移,垃圾文件、旧注册表数据和不必要的后台进程会占用资源并降低性能。幸运的是,许多工具可以让 Windows 保持平稳运行。
Copyright 2014-2025 https://www.php.cn/ All Rights Reserved | php.cn | 湘ICP备2023035733号