• Gitlab: PHP项目CI/CD实践

    目录

    1 说明

    2 CI/CD

    2.1 部署方式一：增量部署

    2.1.1 目标服务器准备 

    2.2.2 Gitlab及Envoy脚本

    2.2 部署方式二：镜像构建与部署

    2.2.1 推送到私有化容器仓库

    准备工作

    脚本

    要点

    2.2.2 推送到hub.docker.com

    准备工作

    脚本

    3 参考：

    1 说明

    • 以一个laravel blog项目为例，做dev分支的CI/CD实践
    • 结合laravel envoy工具做多个远程服务器部署，分两种方式：A. 增量部署 B.镜像构建与部署

    服务器：

    SiteServerIP站点目录
    team1-prj2.dev.iahost001.dev.ia192.168.0.130/www/wwwroot/team1-prj2.dev.ia
    team1-prj2.dev.iahost002.dev.ia192.168.0.131/www/wwwroot/team1-prj2.dev.ia

    2 CI/CD

    2.1 部署方式一：增量部署

    通过Lavavel/Envoy和git拉取新版本文件进行部署

    2.1.1 目标服务器准备 
    • php8.2, 安装所需扩展，务必在cli下测试是否正常, 有些被disable的functions要打开
    • composer self-update， 兼容php8.2
    • 使用Laravel/Envoy分发部署，确保Envoy.blade.php是utf8格式文件
    • 站点目录结构 
      1. root@host001:/www/wwwroot/team1-prj2.dev.ia# tree -d -L 3 ./
      2. ./
      3. ├── current -> /www/wwwroot/team1-prj2.dev.ia/releases/default
      4. ├── releases
      5. │   └── default
      6. │       ├── app
      7. │       ├── bootstrap
      8. │       ├── config
      9. │       ├── database
      10. │       ├── public
      11. │       ├── resources
      12. │       ├── routes
      13. │       ├── storage -> /www/wwwroot/team1-prj2.dev.ia/storage
      14. │       ├── tests
      15. │       └── vendor
      16. └── storage
      17.     ├── app
      18.     │   └── public
      19.     ├── framework
      20.     │   ├── cache
      21.     │   ├── sessions
      22.     │   ├── testing
      23.     │   └── views
      24.     └── logs

    说明：

    team1-prj2.dev.ia应用目录
    team1-prj2.dev.ia/releases版本发布目录，这里只设置了一个default目录，也可根据需要做日期变量发布
    team1-prj2.dev.ia/current

    链接到最新版本，被nginx访问的站点目录路径

    current -> /www/wwwroot/team1-prj2.dev.ia/releases/default/

    team1-prj2.dev.ia/storage

    链接到最新版本的应用数据保存目录，如：日志，缓存等

    storage -> /www/wwwroot/team1-prj2.dev.ia/storage

    team1-prj2.dev.ia/.dev

    .dev文件是运维人员建立的服务器定制环境文件，不进入仓库，链接到项目同名文件

    .env -> /www/wwwroot/team1-prj2.dev.ia/.env*

    team1-prj2.dev.ia/releases/default/.gitlab-ci.ymlgitlab 部署脚本
    team1-prj2.dev.ia/releases/default/Envoy.blade.phpenvoy 部署脚本

    nginx配置

    1. server
    2. {
    3.     listen 80;
    4.     server_name team1-prj2.dev.ia;
    5.     index index.php index.html index.htm default.php default.htm default.html;
    6.     root /www/wwwroot/team1-prj2.dev.ia/current/public;
    7.     #CERT-APPLY-CHECK--START
    8.     # 用于SSL证书申请时的文件验证相关配置 -- 请勿删除
    9.     include /www/server/panel/vhost/nginx/well-known/team1-prj2.dev.ia.conf;
    10.     #CERT-APPLY-CHECK--END
    11.  
    12.     #SSL-START SSL相关配置，请勿删除或修改下一行带注释的404规则
    13.     #error_page 404/404.html;
    14.     #SSL-END
    15.  
    16.     #ERROR-PAGE-START  错误页配置，可以注释、删除或修改
    17.     #error_page 404 /404.html;
    18.     #error_page 502 /502.html;
    19.     #ERROR-PAGE-END
    20.  
    21.     #PHP-INFO-START  PHP引用配置，可以注释或修改
    22.     include enable-php-82.conf;
    23.     #PHP-INFO-END
    24.  
    25.     #REWRITE-START URL重写规则引用,修改后将导致面板设置的伪静态规则失效
    26.     include /www/server/panel/vhost/rewrite/team1-prj2.dev.ia.conf;
    27.     #REWRITE-END
    28.  
    29.     #禁止访问的文件或目录
    30.     location ~ ^/(\.user.ini|\.htaccess|\.git|\.env|\.svn|\.project|LICENSE|README.md)
    31.     {
    32.         return 404;
    33.     }
    34.  
    35.     #一键申请SSL证书验证目录相关设置
    36.     location ~ \.well-known{
    37.         allow all;
    38.     }
    39.  
    40.     #禁止在证书验证目录放入敏感文件
    41.     if ( $uri ~ "^/\.well-known/.*\.(php|jsp|py|js|css|lua|ts|go|zip|tar\.gz|rar|7z|sql|bak)$" ) {
    42.         return 403;
    43.     }
    44.  
    45.     location ~ .*\.(gif|jpg|jpeg|png|bmp|swf)$
    46.     {
    47.         expires      30d;
    48.         error_log /dev/null;
    49.         access_log /dev/null;
    50.     }
    51.  
    52.     location ~ .*\.(js|css)?$
    53.     {
    54.         expires      12h;
    55.         error_log /dev/null;
    56.         access_log /dev/null;
    57.     }
    58.     
    59.     location / {
    60.         try_files $uri $uri/ /index.php?$query_string;
    61.     }
    62.     
    63.     access_log  /www/wwwlogs/team1-prj2.dev.ia.log;
    64.     error_log  /www/wwwlogs/team1-prj2.dev.ia.error.log;
    65. }
    2.2.2 Gitlab及Envoy脚本

    .gitlab-ci.yml

    1. # default:
    2. #  image: edbizarro/gitlab-ci-pipeline-php:7.4
    3. #default:
    4. # image: bennybi/php8.2
    5. # image: bennybi/php7.4
    6.  
    7. stages:
    8.   - test
    9.   - deploy
    10.  
    11. .init_ssh: &init_ssh |
    12.   which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )
    13.   eval $(ssh-agent -s)
    14.   echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
    15.   mkdir -p ~/.ssh
    16.   chmod 700 ~/.ssh
    17.   [[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
    18.  
    19. cache:
    20.   key: ${CI_COMMIT_REF_SLUG}
    21.   paths:
    22.     - vendor/
    23.  
    24. unit_test:
    25.   stage: test
    26.   tags:
    27.     - php
    28.   script:
    29.     - cp .env.test .env
    30.     - composer install
    31.     - composer global require "laravel/envoy"
    32.     - php artisan key:generate
    33.     - php artisan migrate
    34.     - vendor/bin/phpunit
    35.  
    36. deploy_dev:
    37.   stage: deploy
    38.   tags:
    39.     - php
    40.   environment:
    41.     name: dev
    42.     url: http://team1-prj2.dev.ia
    43.   script:
    44.     - *init_ssh
    45.     - vendor/bin/envoy run deploy --branch="$CI_COMMIT_BRANCH" --commit="$CI_COMMIT_SHA"
    46.   rules:
    47.     - if: $CI_COMMIT_BRANCH == "dev"
    48.  
    49. deploy_live:
    50.   stage: deploy
    51.   tags:
    52.     - php
    53.   script:
    54.     - *init_ssh
    55.     - vendor/bin/envoy run deploy --commit="$CI_COMMIT_SHA"
    56.   environment:
    57.     name: live
    58.     url: http://team1-prj2.dev.ia
    59.   when: manual
    60.   rules:
    61.     - if: $CI_COMMIT_BRANCH == "live"

    Envoy.blade.php

    1. @servers(['local' => 'deployer@host001.dev.ia','staging' => 'deployer@host002.dev.ia'])
    2.  
    3. @setup
    4.     $repository = 'git@host001.dev.ia:dev1/team1-prj2.git';
    5.     $releases_dir = '/www/wwwroot/team1-prj2.dev.ia/releases';
    6.     $app_dir = '/www/wwwroot/team1-prj2.dev.ia';
    7.     $release = 'default';
    8.     $new_release_dir = $releases_dir .'/'. $release;
    9.     $user = get_current_user();
    10. @endsetup
    11.  
    12. @story('deploy', ['on' => ['local','staging']])
    13.     sync_repository
    14.     run_composer
    15.     update_symlinks
    16. @endstory
    17.  
    18. @task('sync_repository')
    19.     echo "Current User: {{$user}}, branch:{{$branch}}, commit:{{$commit}}"
    20.     if [ -d "{{$new_release_dir}}" ]; then
    21.         echo 'Pulling repository'
    22.         cd {{ $new_release_dir }}
    23.         git checkout {{ $branch }}
    24.         git fetch
    25.         git reset --hard HEAD
    26.         git merge origin/{{ $branch }}
    27.     else
    28.         echo 'Cloning repository'
    29.         [ -d {{ $releases_dir }} ] || mkdir {{ $releases_dir }}
    30.         git clone --branch {{ $branch }}  --single-branch --depth 1 {{ $repository }} {{ $new_release_dir }}
    31.         cd {{ $new_release_dir }}
    32.         git reset --hard {{ $commit }}
    33.         git config --global --add safe.directory '*'
    34.     fi
    35. @endtask
    36.  
    37. @task('run_composer')
    38.     echo "Starting deployment ({{ $release }})"
    39.     cd {{ $new_release_dir }}
    40.     composer install --prefer-dist --no-scripts -q -o
    41. @endtask
    42.  
    43. @task('update_symlinks')
    44.     if [ ! -d "{{ $app_dir }}/current" ]; then
    45.         echo "Linking storage directory"
    46.         {{-- rm -rf {{ $new_release_dir }}/storage  --}}
    47.         mv {{ $new_release_dir }}/storage {{ $app_dir }}
    48.         ln -nfs {{ $app_dir }}/storage {{ $new_release_dir }}/storage
    49.         
    50.         echo 'Linking .env file'
    51.         ln -nfs {{ $app_dir }}/.env {{ $new_release_dir }}/.env
    52.         
    53.         echo 'Linking current release'
    54.         ln -nfs {{ $new_release_dir }} {{ $app_dir }}/current
    55.         
    56.         chmod 775 -Rf {{ $releases_dir }}
    57.     fi
    58.     chmod 775 -Rf {{ $new_release_dir }}/storage
    59.     {{-- chown deployer:www -Rf {{ $new_release_dir }}/.git  --}}
    60. @endtask
    2.2 部署方式二：镜像构建与部署
    说明：
    1. 应用文件在build_image阶段，打包在镜像文件中，并推送到私有化的项目镜像仓库
    2. lavavel应用所需.env，不进入代码仓库，而是通过 -v 映射到具体路径部署，如示例代码中的 "-v /data0/Projects/team1-prj1/.env:/app/.env"，防止敏感参数外泄
    3. 实例化后作为微服务容器，暴露8181端口提供外部访问
    2.2.1 推送到私有化容器仓库
    准备工作

    -  新建项目team1-prj1，初始化git 仓库为：http://host001.dev.ia:18181/dev1/team1-prj1.git

    -  建好容器仓库（见前文相关部分）

    -  需要在Admin Area->CI/CD->Variables添加docker访问用户变量

    1. $LOCAL_REGISTRY_LOGIN = your docker username
    2. $LOCAL_REGISTRY_PASSWORD
    脚本

     项目中添加Dockerfile 文件，这里用到的原型镜像是我之前定制的php8.2

    1. FROM bennybi/php8.2:latest
    2.  
    3. WORKDIR /app
    4.  
    5. RUN apt-get update -y && apt-get install -y openssl zip unzip git
    6. RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
    7. # RUN docker-php-ext-install pdo mbstring
    8.  
    9. # COPY .env.example .env
    10. COPY . /app
    11. RUN composer install --no-interaction --prefer-dist --optimize-autoloader
    12.  
    13. # Run any additional commands specific to Laravel
    14. RUN php artisan config:cache
    15. RUN php artisan route:cache
    16. RUN php artisan view:cache
    17.  
    18. CMD php artisan serve --host=0.0.0.0 --port=8181
    19. EXPOSE 8181

     .gitlab-ci.yml

    1. default:
    2.   image: docker:19.03.8
    3.   services:
    4.     - mysql:5.7
    5.   before_script:
    6.     - docker info
    7.  
    8. variables:
    9.   # DOCKER_IMAGE_TAG: 'bennybi/team1-prj1'
    10.   APP_NAME: 'team1-prj1'
    11.   REGISTRY_URL: 'http://host001.dev.ia:5050'
    12.   DOCKER_IMAGE_TAG: 'host001.dev.ia:5050/dev1/team1-prj1:1.0.0'
    13.   # DOCKER_IMAGE_TAG: 'host001.dev.ia:5050/dev1/team1-prj1'
    14.   CONTAINER_IMAGE_NAME: ${DOCKER_IMAGE_TAG}
    15.   # MYSQL_DATABASE: test
    16.   # MYSQL_ROOT_PASSWORD: fa843c707ce26702
    17.   # DB_HOST: host001.dev.ia
    18.   # DB_USERNAME: developer
    19.  
    20. stages:
    21.   - test
    22.   - build
    23.   - deploy
    24.  
    25. unit_test:
    26.   stage: test
    27.   tags:
    28.     - php
    29.   script:
    30.     - cp .env.test .env
    31.     - composer install
    32.     - php artisan key:generate
    33.     - php artisan migrate
    34.     - vendor/bin/phpunit
    35.  
    36. build_image:
    37.   stage: build
    38.   tags:
    39.     - php
    40.   script:
    41.     - docker login ${REGISTRY_URL} -u $LOCAL_REGISTRY_LOGIN -p $LOCAL_REGISTRY_PASSWORD
    42.     - docker build --output type=registry,oci-mediatypes=false --cache-from "${DOCKER_IMAGE_TAG}" -t "${DOCKER_IMAGE_TAG}" --push --provenance=false .
    43.     # - docker push ${DOCKER_IMAGE_TAG}:latest
    44.     - docker push ${DOCKER_IMAGE_TAG}
    45.   rules:
    46.     - if: $CI_COMMIT_BRANCH == "dev"
    47.  
    48. deploy_dev:
    49.   stage: deploy
    50.   tags:
    51.     - php
    52.   script:
    53.     - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
    54.     - eval $(ssh-agent -s)
    55.     - ssh-add <(echo "$SSH_PRIVATE_KEY")
    56.     - mkdir -p ~/.ssh
    57.     - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
    58.     - ssh deployer@host002.dev.ia "docker login ${REGISTRY_URL} -u $LOCAL_REGISTRY_LOGIN -p $LOCAL_REGISTRY_PASSWORD && docker pull $CONTAINER_IMAGE_NAME && docker stop ${APP_NAME} || true && docker rm ${APP_NAME} || true && docker run -d -p 8181:8181  -v /data0/Projects/${APP_NAME}/.env:/app/.env ${CONTAINER_IMAGE_NAME}"
    59.   environment:
    60.     name: dev
    61.     url: http://team1-prj1.dev.ia
    62.   # when: manual
    63.   rules:
    64.     - if: $CI_COMMIT_BRANCH == "dev"
    要点

    - 注意build时所需的参数，缺少会诱发错误： “Invalid tag: missing manifest digest”

    We used the buildx flag “–output type=registry,oci-mediatypes=false” to generate a Docker v2.2 manifest list.  We could set the value to “true” and generate an OCI manifest index, but the GitLab UI will incorrectly display “Invalid tag: missing manifest digest”. 

     推送结果图示：

    2.2.2 推送到hub.docker.com
    准备工作

    -  新建项目team1-prj1，初始化git 仓库为：http://host001.dev.ia:18181/dev1/team1-prj1.git

    -  构建的镜像将push到hub.docker.com，因此需要在Admin Area->CI/CD->Variables添加docker访问用户变量

    1. $DOCKER_LOGIN = your docker username
    2. $DOCKER_PASSWORD
    脚本

    Dockerfile 

    同上 ...

    .gitlab-ci.yml

    1. default:
    2.   image: docker:19.03.8
    3.   services:
    4.     - mysql:5.7
    5.   before_script:
    6.     - docker info
    7.  
    8. variables:
    9.   MYSQL_DATABASE: test
    10.   MYSQL_ROOT_PASSWORD: fa843c707ce26702
    11.   DB_HOST: host001.dev.ia
    12.   DB_USERNAME: developer
    13.  
    14. stages:
    15.   - test
    16.   - build
    17.   # - deploy
    18.  
    19. unit_test:
    20.   stage: test
    21.   tags:
    22.     - php
    23.   script:
    24.     - cp .env.test .env
    25.     - composer install
    26.     - php artisan key:generate
    27.     - php artisan migrate
    28.     - vendor/bin/phpunit
    29.  
    30. build_image:
    31.   stage: build
    32.   variables:
    33.     DOCKER_IMAGE_TAG: 'bennybi/team1-prj1'
    34.   tags:
    35.     - php
    36.   script:
    37.     - docker build --cache-from "${DOCKER_IMAGE_TAG}" -t "${DOCKER_IMAGE_TAG}" .
    38.     - docker login --username $DOCKER_LOGIN --password $DOCKER_PASSWORD
    39.     # - docker run my-docker-image /script/to/run/tests
    40.     - docker push ${DOCKER_IMAGE_TAG}:latest
    41.   rules:
    42.     - if: $CI_COMMIT_BRANCH == "dev"
    43.  
    44. # deploy_dev:
    45. #   stage: deploy
    46. #   tags:
    47. #     - php
    48. #   script:
    49. #     - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
    50. #     - eval $(ssh-agent -s)
    51. #     - ssh-add <(echo "$SSH_PRIVATE_KEY")
    52. #     - mkdir -p ~/.ssh
    53. #     - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
    54. #     # - ~/.composer/vendor/bin/envoy run deploy --commit="$CI_COMMIT_SHA"
    55. #   environment:
    56. #     name: dev
    57. #     url: http://team1-prj1.dev.ia
    58. #   when: manual
    59. #   rules:
    60. #     - if: $CI_COMMIT_BRANCH == "dev"

    3 参考：

    - GitLab: automated build and publish of multi-platform container image with GitLab pipeline | Fabian Lee : Software Engineer

    - https://jaumemule.medium.com/build-a-php8-0-fpm-gitlab-ci-cd-application-docker-google-cloud-5f2868e8370

    - https://docs.gitlab.com/ee/ci/docker/using_docker_build.html

     - https://github.com/papertank/envoy-deploy/blob/master/readme.md

    - https://warrickbayman.medium.com/zero-downtime-laravel-deployments-with-envoy-version-2-227c8259e31c

    - Test and deploy Laravel applications with GitLab CI/CD and Envoy | GitLab

    - GitLab CI/CD examples | GitLab

    - 使用GitLab Runner为基于Laravel的PHP项目进行部署 

    - GitLab CI/CD for Beginners [FREE Course] - DEV Community

  • 相关阅读:
    update会锁表吗？
    Tune-A-Video论文阅读
    为什么会突然牙疼？
    关于Spring遇到的各种问题
    企业征信牌照9月末盘点：149家机构荣获上榜，西藏等地机构待批
    如何搭建团队知识库？试试新的工具和方法吧！
    PHP实现DFA算法，查找关键词
    基于持续同调的在线社交网络传播研究
    微信小程序自定义图片上传组件
    [pytorchb笔记]03 --模型
  • 原文地址：https://blog.csdn.net/bennybi/article/details/136363745