• Spring Boot集成tensorflow实现图片检测服务

    1.什么是tensorflow？

    TensorFlow名字的由来就是张量(Tensor)在计算图(Computational Graph)里的流动(Flow)，如图。它的基础就是前面介绍的基于计算图的自动微分，除了自动帮你求梯度之外，它也提供了各种常见的操作(op，也就是计算图的节点)，常见的损失函数，优化算法。

    tensorflow

    • TensorFlow 是一个用于研究和生产的开放源代码机器学习库。TensorFlow 提供了各种 API，可供初学者和专家在桌面、移动、网络和云端环境下进行开发。
    • TensorFlow是采用数据流图（data　flow　graphs）来计算,所以首先我们得创建一个数据流流图,然后再将我们的数据（数据以张量(tensor)的形式存在）放在数据流图中计算. 节点（Nodes）在图中表示数学操作,图中的边（edges）则表示在节点间相互联系的多维数据数组, 即张量（tensor)。训练模型时tensor会不断的从数据流图中的一个节点flow到另一节点, 这就是TensorFlow名字的由来。 张量（Tensor):张量有多种. 零阶张量为 纯量或标量 (scalar) 也就是一个数值. 比如 [1],一阶张量为 向量 (vector), 比如 一维的 [1, 2, 3],二阶张量为 矩阵 (matrix), 比如 二维的 [[1, 2, 3],[4, 5, 6],[7, 8, 9]],以此类推, 还有 三阶 三维的 … 张量从流图的一端流动到另一端的计算过程。它生动形象地描述了复杂数据结构在人工神经网中的流动、传输、分析和处理模式。

    在机器学习中，数值通常由4种类型构成： （1）标量（scalar）：即一个数值，它是计算的最小单元，如“1”或“3.2”等。 （2）向量（vector）：由一些标量构成的一维数组，如[1, 3.2, 4.6]等。 （3）矩阵（matrix）：是由标量构成的二维数组。 （4）张量（tensor）：由多维（通常）数组构成的数据集合，可理解为高维矩阵。

    tensorflow的基本概念

    • 图：描述了计算过程，Tensorflow用图来表示计算过程
    • 张量：Tensorflow 使用tensor表示数据，每一个tensor是一个多维化的数组
    • 操作：图中的节点为op，一个op获得/输入0个或者多个Tensor，执行并计算，产生0个或多个Tensor
    • 会话：session tensorflow的运行需要再绘话里面运行

    tensorflow写代码流程

    • 定义变量占位符
    • 根据数学原理写方程
    • 定义损失函数cost
    • 定义优化梯度下降 GradientDescentOptimizer
    • session 进行训练，for循环
    • 保存saver

    2.环境准备

    整合步骤

    1. 模型构建：首先，我们需要在TensorFlow中定义并训练深度学习模型。这可能涉及选择合适的网络结构、优化器和损失函数等。
    2. 训练数据准备：接下来，我们需要准备用于训练和验证模型的数据。这可能包括数据清洗、标注和预处理等步骤。
    3. REST API设计：为了与TensorFlow模型进行交互，我们需要在SpringBoot中创建一个REST API。这可以使用SpringBoot的内置功能来实现，例如使用Spring MVC或Spring WebFlux。
    4. 模型部署：在模型训练完成后，我们需要将其部署到SpringBoot应用中。为此，我们可以使用TensorFlow的Java API将模型导出为ONNX或SavedModel格式，然后在SpringBoot应用中加载并使用。

    在整合过程中，有几个关键点需要注意。首先，防火墙设置可能会影响TensorFlow训练过程中的网络通信。确保你的防火墙允许TensorFlow访问其所需的网络资源，以免出现训练中断或模型性能下降的问题。其次，要关注版本兼容性。SpringBoot和TensorFlow都有各自的版本更新周期，确保在整合时使用兼容的版本可以避免很多不必要的麻烦。

    模型下载

    模型构建和模型训练这块设计到python代码，这里跳过，感兴趣的可以下载源代码自己训练模型，咱们直接下载训练好的模型

    下载好了，解压放在/resources/inception_v3目录下

    3.代码工程

    实验目的

    实现图片检测

    pom.xml

    1. <?xml version="1.0" encoding="UTF-8"?>
    2. <project xmlns="http://maven.apache.org/POM/4.0.0"
    3.          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    4.          xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    5.     <parent>
    6.         <artifactId>springboot-demo</artifactId>
    7.         <groupId>com.et</groupId>
    8.         <version>1.0-SNAPSHOT</version>
    9.     </parent>
    10.     <modelVersion>4.0.0</modelVersion>
    11.  
    12.     <artifactId>Tensorflow</artifactId>
    13.  
    14.     <properties>
    15.         <maven.compiler.source>11</maven.compiler.source>
    16.         <maven.compiler.target>11</maven.compiler.target>
    17.     </properties>
    18.     <dependencies>
    19.  
    20.         <dependency>
    21.             <groupId>org.springframework.boot</groupId>
    22.             <artifactId>spring-boot-starter-web</artifactId>
    23.         </dependency>
    24.  
    25.         <dependency>
    26.             <groupId>org.springframework.boot</groupId>
    27.             <artifactId>spring-boot-autoconfigure</artifactId>
    28.         </dependency>
    29.         <dependency>
    30.             <groupId>org.springframework.boot</groupId>
    31.             <artifactId>spring-boot-starter-test</artifactId>
    32.             <scope>test</scope>
    33.         </dependency>
    34.         <dependency>
    35.             <groupId>org.tensorflow</groupId>
    36.             <artifactId>tensorflow-core-platform</artifactId>
    37.             <version>0.5.0</version>
    38.         </dependency>
    39.         <dependency>
    40.             <groupId>org.projectlombok</groupId>
    41.             <artifactId>lombok</artifactId>
    42.         </dependency>
    43.  
    44.         <dependency>
    45.             <groupId>jmimemagic</groupId>
    46.             <artifactId>jmimemagic</artifactId>
    47.             <version>0.1.2</version>
    48.         </dependency>
    49.         <dependency>
    50.             <groupId>jakarta.platform</groupId>
    51.             <artifactId>jakarta.jakartaee-api</artifactId>
    52.             <version>9.0.0</version>
    53.         </dependency>
    54.         <dependency>
    55.             <groupId>commons-io</groupId>
    56.             <artifactId>commons-io</artifactId>
    57.             <version>2.16.1</version>
    58.         </dependency>
    59.         <dependency>
    60.             <groupId>org.springframework.restdocs</groupId>
    61.             <artifactId>spring-restdocs-mockmvc</artifactId>
    62.             <scope>test</scope>
    63.         </dependency>
    64.  
    65.     </dependencies>
    66. </project>

    controller

    1. package com.et.tf.api;
    2.  
    3. import java.io.IOException;
    4.  
    5. import com.et.tf.service.ClassifyImageService;
    6. import net.sf.jmimemagic.Magic;
    7. import net.sf.jmimemagic.MagicMatch;
    8. import org.springframework.beans.factory.annotation.Autowired;
    9. import org.springframework.web.bind.annotation.CrossOrigin;
    10. import org.springframework.web.bind.annotation.PostMapping;
    11. import org.springframework.web.bind.annotation.RequestMapping;
    12. import org.springframework.web.bind.annotation.RequestParam;
    13. import org.springframework.web.bind.annotation.RestController;
    14. import org.springframework.web.multipart.MultipartFile;
    15.  
    16. @RestController
    17. @RequestMapping("/api")
    18. public class AppController {
    19.     @Autowired
    20.     ClassifyImageService classifyImageService;
    21.  
    22.  
    23.     @PostMapping(value = "/classify")
    24.     @CrossOrigin(origins = "*")
    25.     public ClassifyImageService.LabelWithProbability classifyImage(@RequestParam MultipartFile file) throws IOException {
    26.         checkImageContents(file);
    27.         return classifyImageService.classifyImage(file.getBytes());
    28.     }
    29.  
    30.     @RequestMapping(value = "/")
    31.     public String index() {
    32.         return "index";
    33.     }
    34.  
    35.     private void checkImageContents(MultipartFile file) {
    36.         MagicMatch match;
    37.         try {
    38.             match = Magic.getMagicMatch(file.getBytes());
    39.         } catch (Exception e) {
    40.             throw new RuntimeException(e);
    41.         }
    42.         String mimeType = match.getMimeType();
    43.         if (!mimeType.startsWith("image")) {
    44.             throw new IllegalArgumentException("Not an image type: " + mimeType);
    45.         }
    46.     }
    47.  
    48. }

    service

    1. package com.et.tf.service;
    2.  
    3. import jakarta.annotation.PreDestroy;
    4. import java.util.Arrays;
    5. import java.util.List;
    6. import lombok.AllArgsConstructor;
    7. import lombok.Data;
    8. import lombok.NoArgsConstructor;
    9. import lombok.extern.slf4j.Slf4j;
    10. import org.springframework.beans.factory.annotation.Value;
    11. import org.springframework.stereotype.Service;
    12. import org.tensorflow.Graph;
    13. import org.tensorflow.Output;
    14. import org.tensorflow.Session;
    15. import org.tensorflow.Tensor;
    16. import org.tensorflow.ndarray.NdArrays;
    17. import org.tensorflow.ndarray.Shape;
    18. import org.tensorflow.ndarray.buffer.FloatDataBuffer;
    19. import org.tensorflow.op.OpScope;
    20. import org.tensorflow.op.Scope;
    21. import org.tensorflow.proto.framework.DataType;
    22. import org.tensorflow.types.TFloat32;
    23. import org.tensorflow.types.TInt32;
    24. import org.tensorflow.types.TString;
    25. import org.tensorflow.types.family.TType;
    26.  
    27. //Inspired from https://github.com/tensorflow/tensorflow/blob/master/tensorflow/java/src/main/java/org/tensorflow/examples/LabelImage.java
    28. @Service
    29. @Slf4j
    30. public class ClassifyImageService {
    31.  
    32.     private final Session session;
    33.     private final List<String> labels;
    34.     private final String outputLayer;
    35.  
    36.     private final int W;
    37.     private final int H;
    38.     private final float mean;
    39.     private final float scale;
    40.  
    41.     public ClassifyImageService(
    42.         Graph inceptionGraph, List<String> labels, @Value("${tf.outputLayer}") String outputLayer,
    43.         @Value("${tf.image.width}") int imageW, @Value("${tf.image.height}") int imageH,
    44.         @Value("${tf.image.mean}") float mean, @Value("${tf.image.scale}") float scale
    45.     ) {
    46.         this.labels = labels;
    47.         this.outputLayer = outputLayer;
    48.         this.H = imageH;
    49.         this.W = imageW;
    50.         this.mean = mean;
    51.         this.scale = scale;
    52.         this.session = new Session(inceptionGraph);
    53.     }
    54.  
    55.     public LabelWithProbability classifyImage(byte[] imageBytes) {
    56.         long start = System.currentTimeMillis();
    57.         try (Tensor image = normalizedImageToTensor(imageBytes)) {
    58.             float[] labelProbabilities = classifyImageProbabilities(image);
    59.             int bestLabelIdx = maxIndex(labelProbabilities);
    60.             LabelWithProbability labelWithProbability =
    61.                 new LabelWithProbability(labels.get(bestLabelIdx), labelProbabilities[bestLabelIdx] * 100f, System.currentTimeMillis() - start);
    62.             log.debug(String.format(
    63.                     "Image classification [%s %.2f%%] took %d ms",
    64.                     labelWithProbability.getLabel(),
    65.                     labelWithProbability.getProbability(),
    66.                     labelWithProbability.getElapsed()
    67.                 )
    68.             );
    69.             return labelWithProbability;
    70.         }
    71.     }
    72.  
    73.     private float[] classifyImageProbabilities(Tensor image) {
    74.         try (Tensor result = session.runner().feed("input", image).fetch(outputLayer).run().get(0)) {
    75.             final Shape resultShape = result.shape();
    76.             final long[] rShape = resultShape.asArray();
    77.             if (resultShape.numDimensions() != 2 || rShape[0] != 1) {
    78.                 throw new RuntimeException(
    79.                     String.format(
    80.                         "Expected model to produce a [1 N] shaped tensor where N is the number of labels, instead it produced one with shape %s",
    81.                         Arrays.toString(rShape)
    82.                     ));
    83.             }
    84.             int nlabels = (int) rShape[1];
    85.             FloatDataBuffer resultFloatBuffer = result.asRawTensor().data().asFloats();
    86.             float[] dst = new float[nlabels];
    87.             resultFloatBuffer.read(dst);
    88.             return dst;
    89.         }
    90.     }
    91.  
    92.     private int maxIndex(float[] probabilities) {
    93.         int best = 0;
    94.         for (int i = 1; i < probabilities.length; ++i) {
    95.             if (probabilities[i] > probabilities[best]) {
    96.                 best = i;
    97.             }
    98.         }
    99.         return best;
    100.     }
    101.  
    102.     private Tensor normalizedImageToTensor(byte[] imageBytes) {
    103.         try (Graph g = new Graph();
    104.              TInt32 batchTensor = TInt32.scalarOf(0);
    105.              TInt32 sizeTensor = TInt32.vectorOf(H, W);
    106.              TFloat32 meanTensor = TFloat32.scalarOf(mean);
    107.              TFloat32 scaleTensor = TFloat32.scalarOf(scale);
    108.         ) {
    109.             GraphBuilder b = new GraphBuilder(g);
    110.             //Tutorial python here: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/examples/label_image
    111.             // Some constants specific to the pre-trained model at:
    112.             // https://storage.googleapis.com/download.tensorflow.org/models/inception_v3_2016_08_28_frozen.pb.tar.gz
    113.             //
    114.             // - The model was trained with images scaled to 299x299 pixels.
    115.             // - The colors, represented as R, G, B in 1-byte each were converted to
    116.             //   float using (value - Mean)/Scale.
    117.  
    118.             // Since the graph is being constructed once per execution here, we can use a constant for the
    119.             // input image. If the graph were to be re-used for multiple input images, a placeholder would
    120.             // have been more appropriate.
    121.             final Output input = b.constant("input", TString.tensorOfBytes(NdArrays.scalarOfObject(imageBytes)));
    122.             final Output output =
    123.                 b.div(
    124.                     b.sub(
    125.                         b.resizeBilinear(
    126.                             b.expandDims(
    127.                                 b.cast(b.decodeJpeg(input, 3), DataType.DT_FLOAT),
    128.                                 b.constant("make_batch", batchTensor)
    129.                             ),
    130.                             b.constant("size", sizeTensor)
    131.                         ),
    132.                         b.constant("mean", meanTensor)
    133.                     ),
    134.                     b.constant("scale", scaleTensor)
    135.                 );
    136.             try (Session s = new Session(g)) {
    137.                 return s.runner().fetch(output.op().name()).run().get(0);
    138.             }
    139.         }
    140.     }
    141.  
    142.     static class GraphBuilder {
    143.         final Scope scope;
    144.  
    145.         GraphBuilder(Graph g) {
    146.             this.g = g;
    147.             this.scope = new OpScope(g);
    148.         }
    149.  
    150.         Output div(Output x, Output y) {
    151.             return binaryOp("Div", x, y);
    152.         }
    153.  
    154.         Output sub(Output x, Output y) {
    155.             return binaryOp("Sub", x, y);
    156.         }
    157.  
    158.         Output resizeBilinear(Output images, Output size) {
    159.             return binaryOp("ResizeBilinear", images, size);
    160.         }
    161.  
    162.         Output expandDims(Output input, Output dim) {
    163.             return binaryOp("ExpandDims", input, dim);
    164.         }
    165.  
    166.         Output cast(Output value, DataType dtype) {
    167.             return g.opBuilder("Cast", "Cast", scope).addInput(value).setAttr("DstT", dtype).build().output(0);
    168.         }
    169.  
    170.         Output decodeJpeg(Output contents, long channels) {
    171.             return g.opBuilder("DecodeJpeg", "DecodeJpeg", scope)
    172.                 .addInput(contents)
    173.                 .setAttr("channels", channels)
    174.                 .build()
    175.                 .output(0);
    176.         }
    177.  
    178.         Output<? extends TType> constant(String name, Tensor t) {
    179.             return g.opBuilder("Const", name, scope)
    180.                 .setAttr("dtype", t.dataType())
    181.                 .setAttr("value", t)
    182.                 .build()
    183.                 .output(0);
    184.         }
    185.  
    186.         private Output binaryOp(String type, Output in1, Output in2) {
    187.             return g.opBuilder(type, type, scope).addInput(in1).addInput(in2).build().output(0);
    188.         }
    189.  
    190.         private final Graph g;
    191.     }
    192.  
    193.     @PreDestroy
    194.     public void close() {
    195.         session.close();
    196.     }
    197.  
    198.     @Data
    199.     @NoArgsConstructor
    200.     @AllArgsConstructor
    201.     public static class LabelWithProbability {
    202.         private String label;
    203.         private float probability;
    204.         private long elapsed;
    205.     }
    206. }

    application.yaml

    1. tf:
    2.     frozenModelPath: inception-v3/inception_v3_2016_08_28_frozen.pb
    3.     labelsPath: inception-v3/imagenet_slim_labels.txt
    4.     outputLayer: InceptionV3/Predictions/Reshape_1
    5.     image:
    6.         width: 299
    7.         height: 299
    8.         mean: 0
    9.         scale: 255
    10.  
    11. logging.level.net.sf.jmimemagic: WARN
    12. spring:
    13.   servlet:
    14.     multipart:
    15.       max-file-size: 5MB

    Application.java

    1. package com.et.tf;
    2.  
    3. import java.io.IOException;
    4. import java.nio.charset.StandardCharsets;
    5. import java.util.List;
    6. import java.util.stream.Collectors;
    7. import lombok.extern.slf4j.Slf4j;
    8. import org.apache.commons.io.IOUtils;
    9. import org.springframework.beans.factory.annotation.Value;
    10. import org.springframework.boot.SpringApplication;
    11. import org.springframework.boot.autoconfigure.SpringBootApplication;
    12. import org.springframework.context.annotation.Bean;
    13. import org.springframework.core.io.ClassPathResource;
    14. import org.springframework.core.io.FileSystemResource;
    15. import org.springframework.core.io.Resource;
    16. import org.tensorflow.Graph;
    17. import org.tensorflow.proto.framework.GraphDef;
    18.  
    19. @SpringBootApplication
    20. @Slf4j
    21. public class Application {
    22.  
    23.     public static void main(String[] args) {
    24.         SpringApplication.run(Application.class, args);
    25.     }
    26.  
    27.     @Bean
    28.     public Graph tfModelGraph(@Value("${tf.frozenModelPath}") String tfFrozenModelPath) throws IOException {
    29.         Resource graphResource = getResource(tfFrozenModelPath);
    30.  
    31.         Graph graph = new Graph();
    32.         graph.importGraphDef(GraphDef.parseFrom(graphResource.getInputStream()));
    33.         log.info("Loaded Tensorflow model");
    34.         return graph;
    35.     }
    36.  
    37.     private Resource getResource(@Value("${tf.frozenModelPath}") String tfFrozenModelPath) {
    38.         Resource graphResource = new FileSystemResource(tfFrozenModelPath);
    39.         if (!graphResource.exists()) {
    40.             graphResource = new ClassPathResource(tfFrozenModelPath);
    41.         }
    42.         if (!graphResource.exists()) {
    43.             throw new IllegalArgumentException(String.format("File %s does not exist", tfFrozenModelPath));
    44.         }
    45.         return graphResource;
    46.     }
    47.  
    48.     @Bean
    49.     public List tfModelLabels(@Value("${tf.labelsPath}") String labelsPath) throws IOException {
    50.         Resource labelsRes = getResource(labelsPath);
    51.         log.info("Loaded model labels");
    52.         return IOUtils.readLines(labelsRes.getInputStream(), StandardCharsets.UTF_8).stream()
    53.             .map(label -> label.substring(label.contains(":") ? label.indexOf(":") + 1 : 0)).collect(Collectors.toList());
    54.     }
    55. }

    以上只是一些关键代码，所有代码请参见下面代码仓库

    代码仓库

    4.测试

    启动 Spring Boot应用程序

    测试图片分类

    访问http://127.0.0.1:8080/，上传一张图片，点击分类

     

    5.引用

     

  • 相关阅读:
    利用API数据接口进行市场调研的详细指南
    抽取泛微和建云的销售合同定时任务（要求记录翻译不成功的字段）
    单例模式、工厂模式 c++关键字 static
    详解数据管理、数据治理、数据资产管理
    java计算机毕业设计旅游管理系统源码+mysql数据库+系统+lw文档+部署
    冰冰学习笔记：Linux下的权限理解
    rust的排序
    PostgreSQL(一) 编译安装运行
    【算法】十一月阳光下的阴影面积
    seq2seq与引入注意力机制的seq2seq
  • 原文地址：https://blog.csdn.net/dot_life/article/details/139818407