百度Apollo自定义安装第三方库（以libtorch为例）

百度Apollo是一个非常优秀的自动驾驶框架，但我们平时在开发中也会遇到各种原repo没有处理的问题。笔者近期想用pytorch的C++前端推理模型，但是遇到了libtorch版本与pytorch版本不匹配的问题，因此想自己安装一个新版本的libtorch。

首先找到Apollo的docker是怎么安装libtorch或其他第三方库的。在/apollo/docker/build/installers文件夹下面，可以看到非常多的install_***.sh，这些就是安装第三方库的脚本。原始的install_libtorch.sh如下

set -e
 
CURR_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
. ${CURR_DIR}/installer_base.sh
 
# TODO(build): Docs on how to build libtorch on Jetson boards
# References:
#   https://github.com/ApolloAuto/apollo/blob/pre6/docker/build/installers/install_libtorch.sh
#   https://github.com/dusty-nv/jetson-containers/blob/master/Dockerfile.pytorch
#   https://forums.developer.nvidia.com/t/pytorch-for-jetson-version-1-6-0-now-available
#   https://github.com/pytorch/pytorch/blob/master/docker/caffe2/ubuntu-16.04-cpu-all-options/Dockerfile
bash ${CURR_DIR}/install_mkl.sh
 
TARGET_ARCH="$(uname -m)"
 
##============================================================##
# libtorch_cpu
 
if [[ "${TARGET_ARCH}" == "x86_64" ]]; then
    # https://download.pytorch.org/libtorch/cpu/libtorch-cxx11-abi-shared-with-deps-1.5.0%2Bcpu.zip
    VERSION="1.7.0-2"
    CHECKSUM="02fd4f30e97ce8911ef933d0516660892392e95e6768b50f591f4727f6224390"
elif [[ "${TARGET_ARCH}" == "aarch64" ]]; then
    VERSION="1.6.0-1"
    CHECKSUM="6d1fba522e746213c209fbf6275fa6bac68e360bcd11cbd4d3bdbddb657bee82"
else
    error "libtorch for ${TARGET_ARCH} not ready. Exiting..."
    exit 1
fi
 
PKG_NAME="libtorch_cpu-${VERSION}-linux-${TARGET_ARCH}.tar.gz"
DOWNLOAD_LINK="https://apollo-system.cdn.bcebos.com/archive/6.0/${PKG_NAME}"
download_if_not_cached "${PKG_NAME}" "${CHECKSUM}" "${DOWNLOAD_LINK}"
 
tar xzf "${PKG_NAME}"
mv "${PKG_NAME%.tar.gz}" /usr/local/libtorch_cpu
rm -f "${PKG_NAME}"
ok "Successfully installed libtorch_cpu ${VERSION}"
 
##============================================================##
# libtorch_gpu
if [[ "${TARGET_ARCH}" == "x86_64" ]]; then
    VERSION="1.7.0-2"
    CHECKSUM="b64977ca4a13ab41599bac8a846e8782c67ded8d562fdf437f0e606cd5a3b588"
    PKG_NAME="libtorch_gpu-${VERSION}-cu111-linux-x86_64.tar.gz"
else # AArch64
    VERSION="1.6.0-1"
    PKG_NAME="libtorch_gpu-1.6.0-1-linux-aarch64.tar.gz"
    CHECKSUM="eeb5a223d9dbe40fe96f16e6711c49a3777cea2c0a8da2445d63e117fdad0385"
fi
 
DOWNLOAD_LINK="https://apollo-system.cdn.bcebos.com/archive/6.0/${PKG_NAME}"
download_if_not_cached "${PKG_NAME}" "${CHECKSUM}" "${DOWNLOAD_LINK}"
 
tar xzf "${PKG_NAME}"
mv "${PKG_NAME%.tar.gz}" /usr/local/libtorch_gpu
 
# Cleanup
rm -f "${PKG_NAME}"
ok "Successfully installed libtorch_gpu ${VERSION}"

 可以看到这一段

PKG_NAME="libtorch_cpu-${VERSION}-linux-${TARGET_ARCH}.tar.gz"

DOWNLOAD_LINK="https://apollo-system.cdn.bcebos.com/archive/6.0/${PKG_NAME}"

download_if_not_cached "${PKG_NAME}" "${CHECKSUM}" "${DOWNLOAD_LINK}"

tar xzf "${PKG_NAME}"
mv "${PKG_NAME%.tar.gz}" /usr/local/libtorch_cpu
rm -f "${PKG_NAME}"
ok "Successfully installed libtorch_cpu ${VERSION}"

实际上就是找到Apollo指定版本的libtorch然后交给Apollo编译，因此想要替换版本的话只需要自己把这几句下载解压之类的东西替换就行。

接下来看笔者自己写的新的install_libtorch_new.sh：

set -e
 
CURR_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
. ${CURR_DIR}/installer_base.sh
 
# TODO(build): Docs on how to build libtorch on Jetson boards
# References:
#   https://github.com/ApolloAuto/apollo/blob/pre6/docker/build/installers/install_libtorch.sh
#   https://github.com/dusty-nv/jetson-containers/blob/master/Dockerfile.pytorch
#   https://forums.developer.nvidia.com/t/pytorch-for-jetson-version-1-6-0-now-available
#   https://github.com/pytorch/pytorch/blob/master/docker/caffe2/ubuntu-16.04-cpu-all-options/Dockerfile
bash ${CURR_DIR}/install_mkl.sh
 
TARGET_ARCH="$(uname -m)"
 
##============================================================##
# libtorch_cpu
 
if [[ "${TARGET_ARCH}" == "x86_64" ]]; then
    # https://download.pytorch.org/libtorch/cpu/libtorch-cxx11-abi-shared-with-deps-1.5.0%2Bcpu.zip
    VERSION="1.7.0-2"
    CHECKSUM="02fd4f30e97ce8911ef933d0516660892392e95e6768b50f591f4727f6224390"
elif [[ "${TARGET_ARCH}" == "aarch64" ]]; then
    VERSION="1.6.0-1"
    CHECKSUM="6d1fba522e746213c209fbf6275fa6bac68e360bcd11cbd4d3bdbddb657bee82"
else
    error "libtorch for ${TARGET_ARCH} not ready. Exiting..."
    exit 1
fi
 
# PKG_NAME="libtorch_cpu-${VERSION}-linux-${TARGET_ARCH}.tar.gz"
# DOWNLOAD_LINK="https://apollo-system.cdn.bcebos.com/archive/6.0/${PKG_NAME}"
# download_if_not_cached "${PKG_NAME}" "${CHECKSUM}" "${DOWNLOAD_LINK}"
PKG_NAME="/apollo/libtorch-cxx11-abi-shared-with-deps-1.10.2+cpu.zip"
unzip "${PKG_NAME}" -d /usr/local/
mv /usr/local/libtorch/ /usr/local/libtorch_cpu
ok "Successfully installed libtorch_cpu ${VERSION}"
 
##============================================================##
# libtorch_gpu
if [[ "${TARGET_ARCH}" == "x86_64" ]]; then
    VERSION="1.7.0-2"
    CHECKSUM="b64977ca4a13ab41599bac8a846e8782c67ded8d562fdf437f0e606cd5a3b588"
    PKG_NAME="libtorch_gpu-${VERSION}-cu111-linux-x86_64.tar.gz"
else # AArch64
    VERSION="1.6.0-1"
    PKG_NAME="libtorch_gpu-1.6.0-1-linux-aarch64.tar.gz"
    CHECKSUM="eeb5a223d9dbe40fe96f16e6711c49a3777cea2c0a8da2445d63e117fdad0385"
fi
 
# DOWNLOAD_LINK="https://apollo-system.cdn.bcebos.com/archive/6.0/${PKG_NAME}"
# download_if_not_cached "${PKG_NAME}" "${CHECKSUM}" "${DOWNLOAD_LINK}"
 
PKG_NAME="/apollo/libtorch-cxx11-abi-shared-with-deps-1.10.2+cu111.zip"
unzip "${PKG_NAME}" -d /usr/local/
mv /usr/local/libtorch/ /usr/local/libtorch_gpu
 
# Cleanup
rm -f "${PKG_NAME}"
ok "Successfully installed libtorch_gpu ${VERSION}"

可以看到首先把下载的命令注释掉了，然后自己写了一个PKG_NAME，这个地方填的是我自己下载的libtorch的zip文件，然后解压到/usr/local目录下，它自己解压出来的是libtorch文件夹，后面我自己分别重命名为了libtorch_cpu和libtorch_gpu文件夹，然后直接

bash docker/build/installer/install_libtorch_new.sh

就会把原来的/usr/local/下面的libtorch_cpu和libtorch_gpu替换成我自己的版本，这里用的是libtorch-cxx11-abi-shared-with-deps-1.10.2+cpu.zip和libtorch-cxx11-abi-shared-with-deps-1.10.2+cu111.zip。

PS：

再说一下pytorch的弱智指引，去他官网上下载的话实际上不能找到所有版本的libtorch，只有这么个结果

 你如果想要跟自己的pytorch版本匹配的话只能自己去下载，我用自己下载的举例。

CPU版本：

https://download.pytorch.org/libtorch/cpu/libtorch-cxx11-abi-shared-with-deps-1.10.2+cpu.zip

这个命名方式很简单，直接把版本号替换就行。 

GPU版本： 

https://download.pytorch.org/libtorch/cu111/libtorch-cxx11-abi-shared-with-deps-1.10.2+cu111.zip gpu版本要先看自己的cuda版本，我的是11.1，所以上面两处都是cu111，然后在改pytorch版本号1.10.2。这么操作就可以下载自己需要的libtorch版本了。