• Ubuntu Linux下安装 TensorFlow等开发环境

    1.  安装基本工具

    1. sudo apt-get install build-essential
    2. sudo apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
    1. sudo apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev libgtkglext1-dev libgtk-3-dev
    2. sudo apt install libcanberra-gtk-module libcanberra-gtk3-module


    2. 安装 OPENMP
        

    sudo apt install libomp-dev

    3. 打开 WayLand    

      编辑 /etc/gdm3/custom.conf，修改配置使得，“WaylandEnable=true”

    1. # edit /etc/gdm3/custom.conf
    2. sudo nano /etc/gdm3/custom.conf
    3. ******************************
    4. + WaylandEnable=true 
    5. ******************************
    6.  
    7. # restart GDM
    8. sudo systemctl restart gdm3

    4. [可选] 安装 SCR

        >> tesseract是一个开源的OCR引擎，最初是由惠普公司开发用来作为其平板扫描仪的 OCR引擎，2005年惠普将其开源出来，之后google接手负责维护    

    1. sudo add-apt-repository ppa:alex-p/tesseract-ocr
    2. sudo apt-get update 
    3. sudo apt-get install tesseract-ocr

        >> 字库下载 : tesseract支持60多种语言的识别不同，使用之前需要先下载对应语言的字库，下载地址：https://github.com/tesseract-ocr/tessdata
        latest update : 2021 .    5 years ago generically 
        >> 下载完成之后把.traineddata字库文件放到tessdata目录下，默认路径是/usr/share/tesseract-ocr /4.00/tessdata    

    1. wget https://gitcode.net/mirrors/tesseract-ocr/tessdata/-/archive/4.1.0/tessdata-4.1.0.tar.gz
    2. tar xf tessdata-4.1.0.tar.gz
    3. sudo cp -a tessdata-4.1.0/*.traineddata /usr/share/tesseract-ocr/4.00/tessdata/

    5.  安装 Intel@ openCL

        ( https://github.com/intel/compute-runtime/releases )
      

    1.  mkdir neo && cd neo
    2. # Download all *.deb packages
    3. wget https://github.com/intel/intel-graphics-compiler/releases/download/igc-1.0.14062.11/intel-igc-core_1.0.14062.11_amd64.deb
    4. wget https://github.com/intel/intel-graphics-compiler/releases/download/igc-1.0.14062.11/intel-igc-opencl_1.0.14062.11_amd64.deb
    5. wget https://github.com/intel/compute-runtime/releases/download/23.22.26516.18/intel-level-zero-gpu-dbgsym_1.3.26516.18_amd64.ddeb
    6. wget https://github.com/intel/compute-runtime/releases/download/23.22.26516.18/intel-level-zero-gpu_1.3.26516.18_amd64.deb
    7. wget https://github.com/intel/compute-runtime/releases/download/23.22.26516.18/intel-opencl-icd-dbgsym_23.22.26516.18_amd64.ddeb
    8. wget https://github.com/intel/compute-runtime/releases/download/23.22.26516.18/intel-opencl-icd_23.22.26516.18_amd64.deb
    9. wget https://github.com/intel/compute-runtime/releases/download/23.22.26516.18/libigdgmm12_22.3.0_amd64.deb 
    10. # Verify sha256 sums for packages
    11. wget https://github.com/intel/compute-runtime/releases/download/23.22.26516.18/ww22.sum
    12. sha256sum -c ww22.sum
    13. # Install all packages as root
    14. sudo dpkg -i *.deb
    15. # In case of installation problems, please install required dependencies, for example:
    16. sudo apt install ocl-icd-libopencl1


    6.  安装 Intel@ oneAPI MKL (Math Kernel Library)

        6.1 在线安装

        ( https://www.intel.com/content/www/us/en/docs/oneapi/installation-guide-linux/2023-0/apt.html )
            

    1. # download the key to system keyring
    2. wget -O- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB | gpg --dearmor | sudo tee /usr/share/keyrings/oneapi-archive-keyring.gpg > /dev/null
    3. # add signed entry to apt sources and configure the APT client to use Intel repository:
    4. echo "deb [signed-by=/usr/share/keyrings/oneapi-archive-keyring.gpg] https://apt.repos.intel.com/oneapi all main" | sudo tee /etc/apt/sources.list.d/oneAPI.list
    5. sudo apt update
    6. sudo apt install intel-basekit 
    7. # Intel® oneAPI HPC Toolkit
    8. sudo apt install intel-hpckit
    9. # Intel® oneAPI IoT Toolkit
    10. sudo apt install intel-iotkit
    11. # Intel® oneAPI DL Framework Developer Toolkit
    12. sudo apt install intel-dlfdkit 
    13. # Intel® AI Analytics Toolkit
    14. sudo apt install intel-aikit
    15. # Intel® oneAPI Rendering Toolkit
    16. sudo apt install intel-renderkit

        6.2 离线安装

      ( https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit-download.html )

    1. wget https://registrationcenter-download.intel.com/akdlm/IRC_NAS/adb8a02c-4ee7-4882-97d6-a524150da358/l_onemkl_p_2023.2.0.49497_offline.sh
    2. sudo sh ./l_onemkl_p_2023.2.0.49497_offline.sh

           +------------------------------------------------------------------------
            | Recommended for host machines with poor or no internet connection
            | Size     913.25 MB
            | Version     2023.2.0
            | Date     July 13, 2023
            | SHA384     f5cc20cdd92ab961693c7649fb0b046937ae8aae92eb1464090a187816e7bad3ccd6ef5bf90924226d5f4d1314fe57ab
            +------------------------------------------------------------------------ 

    7. 编译 open-cv 带 opencv_contrib

    1. # Install minimal prerequisites (Ubuntu 18.04 as reference)
    2. sudo apt update && sudo apt install -y cmake g++ wget unzip
    3. # Download and unpack sources
    4. wget -O opencv.zip https://github.com/opencv/opencv/archive/4.x.zip
    5. wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/4.x.zip
    6. unzip opencv.zip
    7. unzip opencv_contrib.zip
    8. # Create build directory and switch into it
    9. mkdir -p build && cd build
    10. # Configure
    11. cmake -DOPENCV_EXTRA_MODULES_PATH=../opencv_contrib-4.x/modules ../opencv-4.x
    12. # or debug mode
    13. cmake -DCMAKE_BUILD_TYPE=Debug -DOPENCV_EXTRA_MODULES_PATH=../opencv_contrib-4.x/modules ../opencv-4.x
    14. # Build
    15. cmake --build . -j 8

    8. 直接安装预编译好的 TensorFlow

    (https://tensorflow.google.cn/install/source?hl=zh-cn)   

    pip install tensorflow -i https://pypi.tuna.tsinghua.edu.cn/simple

        或    

    pip install tensorflow==2.13 -i https://pypi.tuna.tsinghua.edu.cn/simple

        TensorFlow 将被安装在 ~/.local/lib/python3.8/site-packages/tensorflow 路径下，使用TensorFlow 库的项目将需要包含这个路径 ~/.local/lib/python3.8/site-packages/tensorflow/include as include directory.
        use ~/.local/lib/python3.8/site-packages/tensorflow as lib path
        
          建议添加指向库的软链接，方便部分基于这些库的开源项目的编译。

    1. cd ~/.local/lib/python3.8/site-packages/tensorflow
    2. ln -s libtensorflow_cc.so.2        libtensorflow_cc.so
    3. ln -s libtensorflow_framework.so.2 libtensorflow_framework.so
    4.  
    5. cd ~/.local/lib/python3.8/site-packages/numpy.libs
    6. ln -s libopenblas64_p-r0-15028c96.3.21.so  libopenblas.so
    7. ln -s libquadmath-96973f99.so.0.0.0        libquadmath.so

    9.  手动编译 TenserFlow C++ 库

    （https://blog.csdn.net/MOU_IT/article/details/87976152)

    9.1 install protobuf

    1. wget https://github.com/protocolbuffers/protobuf/releases/download/v3.7.1/protobuf-cpp-3.7.1.tar.gz
    2. tar -xzvf protobuf-cpp-3.7.1.tar.gz
    3. sudo apt-get install automake libtool
    4. ./autogen.sh
    5. ./configure
    6. make
    7. sudo make install
    8. sudo ldconfig
    9. # sudo make uninstall 安装错版本后卸载指令
    10. protoc --version  # 查看protobuf版本

    9.2 Install bazel 

            +--------------------------------+--------------------------+
            |    tensorflow 2.15.0 rc     |    Bazel 6.1.0          |
            |    tensorflow 2.13.0         |    Bazel 5.3.0          |
            +--------------------------------+--------------------------+
            bazel是Google开源的一套编译构建工具，广泛应用于Google内部，包括TensorFlow项目。

    1. # prepare tools
    2. sudo apt-get install pkg-config zip g++ zlib1g-dev unzip python
    3. # download bazel 5.3.0
    4. wget https://github.com/bazelbuild/bazel/releases/download/5.3.0/bazel-5.3.0-installer-linux-x86_64.sh
    5. sudo chmod +x bazel-5.3.0-installer-linux-x86_64.sh
    6. # install bazel to $HOME/.bazel/bin
    7. ./bazel-5.3.0-installer-linux-x86_64.sh --user
    8. export PATH="$PATH:$HOME/bin"

    9.3 downlaod tenserFlow

    1. git clone --recursive https://github.com/tensorflow/tensorflow.git
    2. cd tensorflow
    3. # switch to the branch you want to build
    4. git checkout r2.13  # r1.9, r1.10, etc. 

          

    9.4 build tenserFlow library

    (libtensorflow_cc.so & libtensorflow_cc_framework.so)

    1. ./configure
    2. ## tensorflow:libtensorflow_cc.so
    3. bazel build --config=opt  //tensorflow:libtensorflow_cc.so
    4.  
    5. ## tensorflow:libtensorflow_cc.so with cuda
    6. bazel build --config=opt --config=cuda //tensorflow:libtensorflow_cc.so
    7.  
    8. ## tensorflow/tools/pip_package:build_pip_package
    9. bazel build -c opt --copt=-msse3 --copt=-msse4.1 --copt=-msse4.2 --copt=-mavx --copt=-mavx2 --copt=-mfma //tensorflow:libtensorflow_cc.so
    10.  
    11. bazel build
    12.  
    13. ## build install python-package
    14. bazel-bin/tensorflow/tools/pip_package/build_pip_package package/20230912
    15.  
    16. ## remove old version and install new package
    17. pip uninstall  tensorflow
    18. pip install package/20230912/tensorflow-*.whl

    9.5  安装 eigen

    参考 ： https://eigen.tuxfamily.org/index.php?title=Main_Page

    下载&解压：

    1. wget  https://gitlab.com/libeigen/eigen/-/archive/3.4.0/eigen-3.4.0.tar.bz2
    2. tar xf eigen-3.4.0.tar.bz2

    编译&安装： 

    1. cd eigen-3.4.0
    2. cmake .. && sudo make install

    安装后,头文件安装在/usr/local/include/eigen3/路径下，很多程序中include时 经常使用#include ，而不是使用#include ，所以要建立一个软链接。

    sudo ln -s /usr/local/include/eigen3/Eigen /usr/local/include/Eigen

    9.6 安装 ml_types

         (https://pypi.org/project/ml-dtypes/) (https://github.com/jax-ml/ml_dtypes)
            ml_dtypes is a stand-alone implementation of several NumPy dtype extensions used in machine learning libraries, 

    1. sudo apt install python3-pip
    2. sudo pip install ml_dtypes

     否则将编译 TensorFlow时将报告如下错误： 

         /usr/local/lib/python3.8/dist-packages/third_party/eigen/Eigen
     [ 50%] Building CXX object CMakeFiles/tf_test.dir/src/hello.cpp.o 
      In file included from /home/rd/tensorflow/tensorflow/core/platform/float8.h:19,
       from /home/rd/tensorflow/tensorflow/core/platform/types.h:20,
       from /home/rd/tensorflow/tensorflow/core/platform/env_time.h:20,
       from /home/rd/tensorflow/tensorflow/core/platform/env.h:26,
       from /home/rd/tensorflow-test-prog/src/hello.cpp:1:
       /home/rd/tensorflow/tensorflow/tsl/platform/float8.h:19:10: fatal error: include/float8.h:没有那个文件或目录 
     19 | #include "include/float8.h"  // from @ml_dtypes 
        |          ^~~~~~~~~~~~~~~~~~
     compilation terminated.


    9.7 安装 abseil （absl）

    1. wget https://github.com/abseil/abseil-cpp/archive/refs/tags/20230125.3.tar.gz
    2. tar  xf 20230125.3.tar.gz
    3. cd abseil-cpp-20230125.3
    4. mkdir build
    5. cd build
    6. cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local
    7. make -j8
    8. sudo make install

    否则将编译 TensorFlow时将报告如下错误：

            | In file included from /home/rd/tensorflow/tensorflow/core/platform/cord.h:19,
            |                  from /home/rd/tensorflow/tensorflow/core/platform/tstring.h:19,
            |                  from /home/rd/tensorflow/tensorflow/core/platform/types.h:22,
            |                  from /home/rd/tensorflow/tensorflow/core/platform/env_time.h:20,
            |                  from /home/rd/tensorflow/tensorflow/core/platform/env.h:26,
            |                  from /home/rd/tensorflow-test-prog/src/hello.cpp:1:
            | /home/rd/tensorflow/tensorflow/tsl/platform/cord.h:21:10: fatal error: absl/strings/cord.h: 没有那个文件或目录
            |    21 | #include "absl/strings/cord.h"  // IWYU pragma: export
            |       |          ^~~~~~~~~~~~~~~~~~~~~
     

    9.8 Install protoc & protobuf 

    1. sudo apt-get install autoconf automake libtool curl make g++ unzip
    2. wget https://github.com/protocolbuffers/protobuf/releases/download/v24.2/protobuf-24.2.tar.gz
    3. tar xf protobuf-24.2.tar.gz
    4. cd protobuf-24.2
    5. cp -a abseil-cpp-20230125.3  thirdparty/abseil-cpp
    6. mkdir build && cd build
    7. cmake -Dprotobuf_BUILD_TESTS=OFF ..
    8. make -j8
    9. sudo make install
    10. sudo ldconfig # refresh shared library cache.

    或

    1. wget https://github.com/protocolbuffers/protobuf/releases/download/v24.2/protoc-24.2-linux-x86_64.zip
    2. unzip protoc-24.2-linux-x86_64.zip
    3. sudo cp bin/protoc /usr/local/bin/
    4. sudo cp -a include/google /usr/local/include/

    否则将编译 TensorFlow时将报告如下错误： 

         In file included from /home/rd/tensorflow/tensorflow/tsl/platform/status.h:39,
                      from /home/rd/tensorflow/tensorflow/core/platform/status.h:23,
                      from /home/rd/tensorflow/tensorflow/core/platform/errors.h:27,
                      from /home/rd/tensorflow/tensorflow/core/platform/env.h:27,
                      from /home/rd/tensorflow-test-prog/src/hello.cpp:1:
     /home/rd/tensorflow/tensorflow/tsl/protobuf/error_codes.pb.h:11:10: fatal error: google/protobuf/port_def.inc: 没有那个文件或目录
        11 | #include "google/protobuf/port_def.inc"
           |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     compilation terminated.
     make[2]: *** [CMakeFiles/tf_test.dir/build.make:63：CMakeFiles/tf_test.dir/src/hello.cpp.o] 错误 1
     make[1]: *** [CMakeFiles/Makefile2:76：CMakeFiles/tf_test.dir/all] 错误 2
     make: *** [Makefile:84：all] 错误 2

    9.9    Build error_codes.pb.h & error_codes.pb.cc and so on


            cd ~/tensorflow
            ls tensorflow/tsl/protobuf
                bfc_memory_map.proto  coordination_config.proto   distributed_runtime_payloads.proto  error_codes.proto  rpc_options.proto  test_log.proto
                BUILD                 coordination_service.proto  dnn.proto                           histogram.proto    status.proto
            protoc  --cpp_out=.  tensorflow/tsl/protobuf/*.proto
            ls tensorflow/tsl/protobuf/
                bfc_memory_map.pb.cc       coordination_config.proto           distributed_runtime_payloads.proto  error_codes.proto  rpc_options.proto  test_log.proto
                bfc_memory_map.pb.h        coordination_service.pb.cc          dnn.pb.cc                           histogram.pb.cc    status.pb.cc
                bfc_memory_map.proto       coordination_service.pb.h           dnn.pb.h                            histogram.pb.h     status.pb.h
                BUILD                      coordination_service.proto          dnn.proto                           histogram.proto    status.proto
                coordination_config.pb.cc  distributed_runtime_payloads.pb.cc  error_codes.pb.cc                   rpc_options.pb.cc  test_log.pb.cc
                coordination_config.pb.h   distributed_runtime_payloads.pb.h   error_codes.pb.h                    rpc_options.pb.h   test_log.pb.h
                
            protoc  --cpp_out=.  tensorflow/*/*/*.proto
            protoc  --cpp_out=.  tensorflow/*/*/*/*.proto
            +--------------------------------------------------solve compile error ----------------------------------------------+
            |             [ 50%] Building CXX object CMakeFiles/tf_test.dir/src/hello.cpp.o
            | In file included from /home/rd/tensorflow/tensorflow/core/platform/status.h:23,
            |                  from /home/rd/tensorflow/tensorflow/core/platform/errors.h:27,
            |                  from /home/rd/tensorflow/tensorflow/core/platform/env.h:27,
            |                  from /home/rd/tensorflow-test-prog/src/hello.cpp:1:
            | /home/rd/tensorflow/tensorflow/tsl/platform/status.h:39:10: fatal error: tensorflow/tsl/protobuf/error_codes.pb.h: 没有那个文件或目录
            |    39 | #include "tensorflow/tsl/protobuf/error_codes.pb.h"
            |       |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            | compilation terminated.
            | make[2]: *** [CMakeFiles/tf_test.dir/build.make:63：CMakeFiles/tf_test.dir/src/hello.cpp.o] 错误 1
            | make[1]: *** [CMakeFiles/Makefile2:76：CMakeFiles/tf_test.dir/all] 错误 2
            | make: *** [Makefile:84：all] 错误 2
            +--------------------------------------------------------------------------------------------------------------------+
            [ 50%] Building CXX object CMakeFiles/tf_test.dir/src/hello.cpp.o
            | In file included from /home/rd/tensorflow-test-prog/src/hello.cpp:2:
            | /home/rd/tensorflow/tensorflow/core/public/session.h:24:10: fatal error: tensorflow/core/framework/device_attributes.pb.h: 没有那个文件或目录
             |   24 | #include "tensorflow/core/framework/device_attributes.pb.h"
            |       |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            | compilation terminated.
            | make[2]: *** [CMakeFiles/tf_test.dir/build.make:63：CMakeFiles/tf_test.dir/src/hello.cpp.o] 错误 1
            | make[1]: *** [CMakeFiles/Makefile2:76：CMakeFiles/tf_test.dir/all] 错误 2
            | make: *** [Makefile:84：all] 错误 2
            +--------------------------------------------------------------------------------------------------------------------+
            

    9.10   创建指向 libtensorflow_framework.so.2.15.0 的符号链接

            cd ~/tensorflow/bazel-bin/tensorflow
            ln -s libtensorflow_framework.so.2.15.0 libtensorflow_framework.so
            ln -s libtensorflow_framework.so.2.15.0 libtensorflow_framework.so.2
                lrwxrwxrwx  1 rd rd        33 9月   6 16:43 libtensorflow_framework.so -> libtensorflow_framework.so.2.15.0
                lrwxrwxrwx  1 rd rd        33 9月   6 16:43 libtensorflow_framework.so.2 -> libtensorflow_framework.so.2.15.0
            +--------------------------------------------------solve compile error ----------------------------------------------+
            |     -- Build files have been written to: /home/rd/tensorflow-test-prog
            | [ 50%] Building CXX object CMakeFiles/tf_test.dir/src/hello.cpp.o
            | [100%] Linking CXX executable tf_test
            | /usr/bin/ld: 找不到 -ltensorflow_framework
            | collect2: error: ld returned 1 exit status
            | make[2]: *** [CMakeFiles/tf_test.dir/build.make:84：tf_test] 错误 1
            | make[1]: *** [CMakeFiles/Makefile2:76：CMakeFiles/tf_test.dir/all] 错误 2
            | make: *** [Makefile:84：all] 错误 2s
            +--------------------------------------------------------------------------------------------------------------------+


     

  • 相关阅读:
    Spring复习——day16_SpringMVC_异常处理器
    Spring Boot中配置文件生效位置
    Vue2中10种组件通信方式和实践技巧
    python 查找波峰和波谷
    个人申请软著需要具备什么
    猿创征文｜云原生|kubernetes学习之RBAC（六）
    【C++】：在构造函数中使用new时的注意事项
    git学习
    python核心编程|一本书，玩转python！让你从入门到大牛！
    云原生：10分钟了解一下Kubernetes架构
  • 原文地址：https://blog.csdn.net/Tonyfield/article/details/133283002