• 世上最全PETSC Linux安装攻略：天河新一代超算上PETSC安装，运行

    1.  make ex9
    2. /thfs1/software/mpich/mpi-n-gcc9.3.0/bin/mpicc -Wall -Wwrite-strings -Wno-unknown-pragmas -Wno-lto-type-mismatch -fstack-protector -fvisibility=hidden -g3 -O0  -I/thfs1/home/***/petsc-3.18.1/include -I/thfs1/home/***/petsc-3.18.1/arch-linux-c-debug/include -I/thfs1/software/mpich/mpi-n-gcc9.3.0/include     -export-dynamic ex9.c  -Wl,-rpath,/thfs1/home/***/petsc-3.18.1/arch-linux-c-debug/lib -L/thfs1/home/***/petsc-3.18.1/arch-linux-c-debug/lib -Wl,-rpath,/thfs1/home/***/petsc-3.18.1/arch-linux-c-debug/lib -L/thfs1/home/***/petsc-3.18.1/arch-linux-c-debug/lib -Wl,-rpath,/thfs1/software/mpich/mpi-n-gcc9.3.0/lib -L/thfs1/software/mpich/mpi-n-gcc9.3.0/lib -Wl,-rpath,/thfs1/software/spack/optv018/linux-ubuntu20.04-aarch64/gcc-8.5.0/gcc-9.3.0-g3g7tub/lib64 -Wl,-rpath,/thfs1/software/spack/optv018/linux-ubuntu20.04-aarch64/gcc-8.5.0/gcc-9.3.0-g3g7tub/lib/gcc/aarch64-unknown-linux-gnu/9.3.0 -L/thfs1/software/spack/optv018/linux-ubuntu20.04-aarch64/gcc-8.5.0/gcc-9.3.0-g3g7tub/lib/gcc/aarch64-unknown-linux-gnu/9.3.0 -L/thfs1/software/spack/optv018/linux-ubuntu20.04-aarch64/gcc-8.5.0/gcc-9.3.0-g3g7tub/lib64 -Wl,-rpath,/lib/aarch64-linux-gnu -L/lib/aarch64-linux-gnu -Wl,-rpath,/usr/lib/aarch64-linux-gnu -L/usr/lib/aarch64-linux-gnu -Wl,-rpath,/thfs1/software/spack/optv018/linux-ubuntu20.04-aarch64/gcc-8.5.0/gcc-9.3.0-g3g7tub/lib -L/thfs1/software/spack/optv018/linux-ubuntu20.04-aarch64/gcc-8.5.0/gcc-9.3.0-g3g7tub/lib -lpetsc -lflapack -lfblas -lpthread -lm -lX11 -lstdc++ -ldl -lmpifort -lmpi -lgfortran -lm -lgfortran -lm -lgcc_s -lstdc++ -ldl -o ex9

    1) 下载PETSC

    https://petsc.org/release/install/download/

    目前最新版是3.18.1

    https://ftp.mcs.anl.gov/pub/petsc/release-snapshots/petsc-3.18.1.tar.gz

    2）PETSC依赖lapack，blas和mpi

    下载

    在https://bitbucket.org/petsc/pkg-fblaslapack/downloads/中下载 

    https://bitbucket.org/petsc/pkg-fblaslapack/get/e8a03f57d64c.zip  

    具体名字可能会变

    下载得到：petsc-pkg-fblaslapack-e8a03f57d64c.zip

    MPI天河超算已经有，使用

    $ module avail | grep mpi-n 得到 
    mpich-3.4.2-gcc-7.5.0-jndkedx
    mpich-3.4.2-gcc-9.3.0-fy2diix
    py-mpi4py-3.1.2-gcc-9.3.0-6zi3t5f

    $ module load module load mpich/mpi-n-gcc9.3.0

    $ which mpif90

    /thfs1/software/mpich/mpi-n-gcc9.3.0/bin/mpif90

    得到具体地址，会自动把库和头文件路径加到当前界面的系统环境中

    3）

    把petsc-3.18.1.tar.gz上传到登录节点

    1. gunzip petsc-3.18.1.tar.gz
    2. tar -xf petsc-3.18.1

    得到解压后的目录petsc-3.18.1

    把petsc-pkg-fblaslapack-e8a03f57d64c.zip上传到登录节点

    改名成：petsc-pkg-fblaslapack.zip

    unzip petsc-pkg-fblaslapack.zip

    解压后移动到目录petsc-3.18.1

    3）

    cd到目录 petsc-3.18.1

    1. export PETSC_DIR=$PWD
    2. mkdir arch-linux-c-debug
    3. export PETSC_ARCH=arch-linux-c-debug
    4. ./configure --with-mpi-dir=/thfs1/software/mpich/mpi-n-gcc9.3.0/ --download-fblaslapack=/thfs1/home/***/petsc-3.18.1/petsc-pkg-fblaslapack

    ***指特定用户目录

    这个过程挺慢的，等了好长时间。

    这是什么玩意? Trying to download ./petsc-pkg-fblaslapack for FBLASLAPACK

    不是已经下载了吗？

    是这个编译器吗？

    Found environment variable:
      F77=/thfs1/software/spack/optv018/linux-ubuntu20.04-aarch64/gcc-8.5.0/gcc-9.3.0-g3g7tub/bin/gfortran.

    这是全部的输出：

    1. ==========================================================================================
    2.                          Configuring PETSc to compile on your system
    3. ==========================================================================================
    4. ==========================================================================================
    5.                                      ***** WARNING *****
    6.   Found environment variable:
    7.   CC=/thfs1/software/spack/optv018/linux-ubuntu20.04-aarch64/gcc-8.5.0/gcc-9.3.0-g3g7tub/bin/gcc.
    8.   Ignoring it! Use "./configure CC=$CC" if you really want to use this value
    9. ==========================================================================================
    10. ==========================================================================================
    11.                                      ***** WARNING *****
    12.   Found environment variable:
    13.   CXX=/thfs1/software/spack/optv018/linux-ubuntu20.04-aarch64/gcc-8.5.0/gcc-9.3.0-g3g7tub/bin/g++.
    14.   Ignoring it! Use "./configure CXX=$CXX" if you really want to use this value
    15. ==========================================================================================
    16. ==========================================================================================
    17.                                      ***** WARNING *****
    18.   Found environment variable:
    19.   FC=/thfs1/software/spack/optv018/linux-ubuntu20.04-aarch64/gcc-8.5.0/gcc-9.3.0-g3g7tub/bin/gfortran.
    20.   Ignoring it! Use "./configure FC=$FC" if you really want to use this value
    21. ==========================================================================================
    22. ==========================================================================================
    23.                                      ***** WARNING *****
    24.   Found environment variable:
    25.   F77=/thfs1/software/spack/optv018/linux-ubuntu20.04-aarch64/gcc-8.5.0/gcc-9.3.0-g3g7tub/bin/gfortran.
    26.   Ignoring it! Use "./configure F77=$F77" if you really want to use this value
    27. ==========================================================================================
    28. ==========================================================================================
    29.                   Trying to download ./petsc-pkg-fblaslapack for FBLASLAPACK
    30. ==========================================================================================
    31. ==========================================================================================
    32.                      Compiling FBLASLAPACK; this may take several minutes
    33. ==========================================================================================
    34. Compilers:
    35.   C Compiler:         /thfs1/software/mpich/mpi-n-gcc9.3.0/bin/mpicc  -Wall -Wwrite-strings -Wno-unknown-pragmas -Wno-lto-type-mismatch -fstack-protector -fvisibility=hidden -g3 -O0
    36.     Version: gcc (Spack GCC) 9.3.0
    37.   C++ Compiler:         /thfs1/software/mpich/mpi-n-gcc9.3.0/bin/mpicxx  -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -Wno-lto-type-mismatch -fstack-protector -fvisibility=hidden -g -O0  -std=gnu++17
    38.     Version: g++ (Spack GCC) 9.3.0
    39.   Fortran Compiler:         /thfs1/software/mpich/mpi-n-gcc9.3.0/bin/mpif90  -Wall -ffree-line-length-none -ffree-line-length-0 -Wno-lto-type-mismatch -Wno-unused-dummy-argument -g -O0
    40.     Version: GNU Fortran (Spack GCC) 9.3.0
    41. Linkers:
    42.   Shared linker:   /thfs1/software/mpich/mpi-n-gcc9.3.0/bin/mpicc  -shared  -Wall -Wwrite-strings -Wno-unknown-pragmas -Wno-lto-type-mismatch -fstack-protector -fvisibility=hidden -g3 -O0
    43.   Dynamic linker:   /thfs1/software/mpich/mpi-n-gcc9.3.0/bin/mpicc  -shared  -Wall -Wwrite-strings -Wno-unknown-pragmas -Wno-lto-type-mismatch -fstack-protector -fvisibility=hidden -g3 -O0
    44.   Libraries linked against:   -lstdc++ -ldl
    45. BlasLapack:
    46.   Libraries:  -Wl,-rpath,arch-linux-c-debug/lib -Larch-linux-c-debug/lib -lflapack -lfblas
    47.   uses 4 byte integers
    48. MPI:
    49.   Version:    3
    50.   Includes:   -I/thfs1/software/mpich/mpi-n-gcc9.3.0/include
    51.   mpiexec: /thfs1/software/mpich/mpi-n-gcc9.3.0/bin/mpiexec
    52.   Implementation: mpich3
    53.   MPICH_NUMVERSION: 30402300
    54. X:
    55.   Libraries:  -lX11
    56. pthread:
    57.   Libraries:  -lpthread
    58. fblaslapack:
    59. cmake:
    60.   Version:    3.23.1
    61.   Executable: /thfs1/software/spack/optv018/linux-ubuntu20.04-aarch64/gcc-9.3.0/cmake-3.23.1-2yky63z/bin/cmake
    62. python:
    63.   Version:    3.9.12
    64.   Executable: /thfs1/software/spack/optv018/linux-ubuntu20.04-aarch64/gcc-9.3.0/python-3.9.12-6onwaq6/bin/python3
    65. regex:
    66. bison:
    67.   Version:    3.5
    68.   Executable: /usr/bin/bison
    69. PETSc:
    70.   Language used to compile PETSc: C
    71.   PETSC_ARCH: arch-linux-c-debug
    72.   PETSC_DIR: /thfs1/home/***/petsc-3.18.1
    73.   Prefix: <inplace installation>
    74.   Scalar type: real
    75.   Precision: double
    76.   Support for __fp16
    77.   Integer size: 4 bytes
    78.   Single library: yes
    79.   Shared libraries: yes
    80.   Memory alignment from malloc(): 16 bytes
    81.   Using GNU make: /usr/bin/gmake
    82. xxx=========================================================================xxx
    83.  Configure stage complete. Now build PETSc libraries with:
    84.    make PETSC_DIR=/thfs1/home/***/petsc-3.18.1 PETSC_ARCH=arch-linux-c-debug all
    85. xxx====================================================================

    make PETSC_DIR=/thfs1/home/***/petsc-3.18.1 PETSC_ARCH=arch-linux-c-debug all

    1. ==========================================
    2.           FC arch-linux-c-debug/obj/sys/fsrc/somefort.o
    3.           FC arch-linux-c-debug/obj/sys/f90-src/fsrc/f90_fwrap.o
    4.           CC arch-linux-c-debug/obj/sys/classes/random/interface/dlregisrand.o
    5.           CC arch-linux-c-debug/obj/sys/classes/random/interface/ftn-auto/randomcf.o
    6.           CC arch-linux-c-debug/obj/sys/classes/random/interface/ftn-auto/randomf.o
    7.           CC arch-linux-c-debug/obj/sys/classes/random/interface/randreg.o
    8.           CC arch-linux-c-debug/obj/sys/classes/viewer/interface/flush.o
    9.           CC arch-linux-c-debug/obj/sys/classes/random/impls/rand/rand.o
    10.           CC arch-linux-c-debug/obj/sys/classes/random/impls/rander48/rander48.o
    11. 		  。。。。。。
    12. 		  
    13.           FC arch-linux-c-debug/obj/snes/f90-mod/petscsnesmod.o
    14.           FC arch-linux-c-debug/obj/ts/f90-mod/petsctsmod.o
    15.           FC arch-linux-c-debug/obj/tao/f90-mod/petsctaomod.o
    16.      CLINKER arch-linux-c-debug/lib/libpetsc.so.3.18.1
    17. =========================================
    18. Now to check if the libraries are working do:
    19. make PETSC_DIR=/thfs1/home/***/petsc-3.18.1 PETSC_ARCH=arch-linux-c-debug check
    20. =========================================

    最后 告诉我们怎么测试，但是输出真是太长了，总共1400多行。

    测试是否安装成功

    make PETSC_DIR=/thfs1/home/***/petsc-3.18.1 PETSC_ARCH=arch-linux-c-debug check

    结果如下：

    1. Running check examples to verify correct installation
    2. Using PETSC_DIR=/thfs1/home/***/petsc-3.18.1 and PETSC_ARCH=arch-linux-c-debug
    3. C/C++ example src/snes/tutorials/ex19 run successfully with 1 MPI process
    4. C/C++ example src/snes/tutorials/ex19 run successfully with 2 MPI processes
    5. Fortran example src/snes/tutorials/ex5f run successfully with 1 MPI process
    6. Completed test examples

    测试成功。

    4）案例测试

    cd 到src/ksp/ksp/tutorials

    make ex9

    得到 如下 

     make ex9
    /thfs1/software/mpich/mpi-n-gcc9.3.0/bin/mpicc -Wall -Wwrite-strings -Wno-unknown-pragmas -Wno-lto-type-mismatch -fstack-protector -fvisibility=hidden -g3 -O0  -I/thfs1/home/***/petsc-3.18.1/include -I/thfs1/home/***/petsc-3.18.1/arch-linux-c-debug/include -I/thfs1/software/mpich/mpi-n-gcc9.3.0/include     -export-dynamic ex9.c  -Wl,-rpath,/thfs1/home/***/petsc-3.18.1/arch-linux-c-debug/lib -L/thfs1/home/***/petsc-3.18.1/arch-linux-c-debug/lib -Wl,-rpath,/thfs1/home/***/petsc-3.18.1/arch-linux-c-debug/lib -L/thfs1/home/***/petsc-3.18.1/arch-linux-c-debug/lib -Wl,-rpath,/thfs1/software/mpich/mpi-n-gcc9.3.0/lib -L/thfs1/software/mpich/mpi-n-gcc9.3.0/lib -Wl,-rpath,/thfs1/software/spack/optv018/linux-ubuntu20.04-aarch64/gcc-8.5.0/gcc-9.3.0-g3g7tub/lib64 -Wl,-rpath,/thfs1/software/spack/optv018/linux-ubuntu20.04-aarch64/gcc-8.5.0/gcc-9.3.0-g3g7tub/lib/gcc/aarch64-unknown-linux-gnu/9.3.0 -L/thfs1/software/spack/optv018/linux-ubuntu20.04-aarch64/gcc-8.5.0/gcc-9.3.0-g3g7tub/lib/gcc/aarch64-unknown-linux-gnu/9.3.0 -L/thfs1/software/spack/optv018/linux-ubuntu20.04-aarch64/gcc-8.5.0/gcc-9.3.0-g3g7tub/lib64 -Wl,-rpath,/lib/aarch64-linux-gnu -L/lib/aarch64-linux-gnu -Wl,-rpath,/usr/lib/aarch64-linux-gnu -L/usr/lib/aarch64-linux-gnu -Wl,-rpath,/thfs1/software/spack/optv018/linux-ubuntu20.04-aarch64/gcc-8.5.0/gcc-9.3.0-g3g7tub/lib -L/thfs1/software/spack/optv018/linux-ubuntu20.04-aarch64/gcc-8.5.0/gcc-9.3.0-g3g7tub/lib -lpetsc -lflapack -lfblas -lpthread -lm -lX11 -lstdc++ -ldl -lmpifort -lmpi -lgfortran -lm -lgfortran -lm -lgcc_s -lstdc++ -ldl -o ex9

    生成可执行文件 ex9

    因为ex9没有输出， 同样可以make ex1

    1. yhrun -n 1 -p thcp1 ex1
    2.  
    3. KSP Object: 1 MPI process
    4.   type: gmres
    5.     restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement
    6.     happy breakdown tolerance 1e-30
    7.   maximum iterations=10000, initial guess is zero
    8.   tolerances:  relative=1e-05, absolute=1e-50, divergence=10000.
    9.   left preconditioning
    10.   using PRECONDITIONED norm type for convergence test
    11. PC Object: 1 MPI process
    12.   type: jacobi
    13.     type DIAGONAL
    14.   linear system matrix = precond matrix:
    15.   Mat Object: 1 MPI process
    16.     type: seqaij
    17.     rows=10, cols=10
    18.     total: nonzeros=28, allocated nonzeros=50
    19.     total number of mallocs used during MatSetValues calls=0
    20.       not using I-node routines
    21. Norm of error 2.41202e-15, Iterations 5

    5）yhrun多进程运行及出现的问题

    好像是MPI不匹配

    1.  yhrun -n 2 -p thcp1 ex1
    2. [0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
    3. [0]PETSC ERROR: Example/application run with number of MPI ranks it does not support
    4. [0]PETSC ERROR: This is a uniprocessor example only!
    5. [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
    6. [0]PETSC ERROR: Petsc Release Version 3.18.1, Oct 26, 2022
    7. [0]PETSC ERROR: ex1 on a arch-linux-c-debug named cn309 by monkeycode Sat Nov 19 10:57:32 2022
    8. [0]PETSC ERROR: Configure options --with-mpi-dir=/***/mpich/mpi-n-gcc9.3.0/ --download-fblaslapack=/thfs1/home/***/petsc-3.18.1/petsc-pkg-fblaslapack
    9. [0]PETSC ERROR: #1 main() at ex1.c:30
    10. [0]PETSC ERROR: No PETSc Option Table entries
    11. [0]PETSC ERROR: ----------------End of Error Message -------send entire error message to petsc-maint@mcs.anl.gov----------
    12. application called MPI_Abort(MPI_COMM_WORLD, 94) - process 0
    13. slurmstepd: error: *** STEP 1706171.0 ON cn309 CANCELLED AT 2022-11-19T10:57:32 ***
    14. srun: Job step aborted: Waiting up to 32 seconds for job step to finish.
    15. srun: error: cn309: tasks 0-1: Killed

    感觉是这个地方 不对：--with-mpi-dir

     rm arch-linux-c-debug/* -rf

    把上面的过程重新来一遍

    可能是问题本身的问题：

    1. cd petsc-3.18.1/src/snes/tutorials
    2.  
    3. make ex19
    4.  
    5.  yhrun -n 4 -p thcp1 ./ex19
    6.  
    7. srun: job 1706249 queued and waiting for resources
    8. srun: job 1706249 has been allocated resources
    9. lid velocity = 0.0625, prandtl # = 1., grashof # = 1.
    10. Number of SNES iterations = 2

    但是yhrun -n 5 -p thcp1 ./ex19

    就会出错：

    1. yhrun -n 5 -p thcp1 ./ex19
    2. srun: job 1706250 queued and waiting for resources
    3. srun: job 1706250 has been allocated resources
    4. [0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
    5. [0]PETSC ERROR: Argument out of range
    6. [0]PETSC ERROR: Partition in y direction is too fine! 4 5
    7. [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
    8. [0]PETSC ERROR: Petsc Release Version 3.18.1, Oct 26, 2022
    9. [0]PETSC ERROR: /thfs1/home/***/petsc-3.18.1/src/snes/tutorials/./ex19 on a arch-linux-c-debug named cn291 by monkeycode Sat Nov 19 11:54:26 2022
    10. [0]PETSC ERROR: Configure options --with-mpi-dir=/thfs1/software/mpich/mpi-n-gcc9.3.0/ --download-fblaslapack=/thfs1/home/***/petsc-3.18.1/petsc-pkg-fblaslapack
    11. [0]PETSC ERROR: #1 DMSetUp_DA_2D() at /thfs1/home/***/petsc-3.18.1/src/dm/impls/da/da2.c:258
    12. [0]PETSC ERROR: #2 DMSetUp_DA() at /thfs1/home/***/petsc-3.18.1/src/dm/impls/da/dareg.c:21
    13. [0]PETSC ERROR: #3 DMSetUp() at /thfs1/home/***/petsc-3.18.1/src/dm/interface/dm.c:824
    14. [0]PETSC ERROR: #4 main() at ex19.c:110
    15. [0]PETSC ERROR: No PETSc Option Table entries
    16. [0]PETSC ERROR: ----------------End of Error Message -------send entire error message to petsc-maint@mcs.anl.gov----------
    17. application called MPI_Abort(MPI_COMM_SELF, 63) - process 0
    18. slurmstepd: error: *** STEP 1706250.0 ON cn291 CANCELLED AT 2022-11-19T11:54:26 ***
    19. srun: Job step aborted: Waiting up to 32 seconds for job step to finish.
    20. srun: error: cn291: tasks 0-4: Killed

    应该是正确 的了

    6)运行时间比较久的示例：

    ./ex45 -ksp_monitor_short -da_grid_x 81 -da_grid_y 81 -da_grid_z 81 -pc_type mg -pc_mg_levels 3 -mg_levels_ksp_type richardson -mg_levels_ksp_max_it 1 -mg_levels_pc_type bjacobi

    -da_grid_x 81 -da_grid_y 81 -da_grid_z 81，这个81可以是21，41等，具体自己试一试

  • 相关阅读:
    计算机网络基础知识总结，就这一篇了！
    Tableau可视化项目
    LeetCode-47-全排列Ⅱ
    AI 辅助学 Java | 专栏 1 帮你学 Java
    基于HBuilderX+UniApp+ThorUI的手机端前端的页面组件化开发经验
    Java-基本数据类型
    Tomcat的安装与使用,maven与Servlet的使用
    并发基础（四）：线程池
    【校招VIP】java语言类和对象之map、set集合
    Iterator-Generator 迭代器和生成器
  • 原文地址：https://blog.csdn.net/anlongstar/article/details/127933359