下机数据自动处理By peaktrace with Docker

发表于 | 更新于
本文字数： 6.2k | 阅读时长 ≈ 6 分钟
Centos7.2版本较低，通过docker-compose构建时候会出现如下错误：Error response from daemon: OCI runtime create failed
建议Centos7.4以上，或者通过yum update升级
安装Docker
旧的版本
yum remove docker docker-client docker-client-latest docker-common docker-latest docker-latest-logrotate docker-logrotate docker-selinux docker-engine-selinux docker-engine
安装依赖
yum install -y yum-utils device-mapper-persistent-data lvm2
添加安装源
yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
安装Docker
yum install docker-ce
启动Docker
systemctl start docker
安装Docker-compose
curl -L "https://github.com/docker/compose/releases/download/1.23.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
Docker-compose构建

注意事项
执行docker-compose命令时，务必进入docker-compose.yml所在的目录，否则无法正常执行。

上传项目至服务器

路径：/opt/clims

配置docker-compose.yml

注意事项
宿主机下机数据路径务必与容器内部路径一致,请根据实际情况修改!

upload successful

version: ‘3’
services:
    db:
      container_name: mysql
      image: mysql:5.6
      environment:
        MYSQL_ROOT_PASSWORD: xxxxxxxx
      ports:
        - “3306:3306”
      volumes:
        - ./mysql/data:/var/lib/mysql
 
    web:
        container_name: nginx
        build: ./nginx
        #image: nginx:test
        restart: always
        ports:
        - “80:80”
        volumes:
        - ./nginx/conf:/opt/nginx/conf
        - ./django/clims:/opt/clims-status
        depends_on:
        - status
 
    status:
        container_name: django
        build: ./django
        #image: django:test
        restart: always
        # command: uwsgi  –emperor uwsgi.ini
        command: uwsgi –ini /opt/clims/uwsgi/uwsgi.ini
        ports:
         - “8001:8001”
        volumes:
        - ./django/clims:/opt/clims
        depends_on:
         - db
 
    pipeline:
        container_name: pipeline
        build: ./pipeline
        restart: always
        volumes:
        - ./pipeline/internal:/opt/pipeline/internal
        - ./pipeline/external:/opt/pipeline/external
        - ./pipeline/cron:/var/spool/cron
        - /sanger-pipeline/clims-in-uat:/sanger-pipeline/clims-in-uat
        - /sanger-pipeline/clims-out-uat:/sanger-pipeline/clims-out-uat
        depends_on:
         - db
         - status
         - web
 
volumes:
    data:
    clims:
    conf:

开始构建
docker-compose up -d

备注
执行此命令会自动构建镜像，并以此镜像自动创建容器，至此已完成自动化部署，可以访问http://宿主机IP/即可！

查看镜像

upload successful

上传镜像至本地仓库

  • 苏州Harbor仓库地址：https://xx.local.genewiz.com

// 登录本地仓库
docker login https://xx.local.genewiz.com

upload successful

// 本地镜像标记TAG
docker tag clims_pipeline:latest xx.local.genewiz.com/clims/pipeline:latest
docker tag clims_web:latest xx.local.genewiz.com/clims/web:latest
docker tag clims_status:latest xx.local.genewiz.com/clims/status:latest
docker tag mysql:5.6 xx.local.genewiz.com/clims/mysql:5.6
// 上传
docker push xx.local.genewiz.com/clims/pipeline:latest
docker push xx.local.genewiz.com/clims/web:latest
docker push xx.local.genewiz.com/clims/status:latest
docker push xx.local.genewiz.com/clims/mysql:5.6

登录仓库查看上传镜像

upload successful

注意事项

执行docker-compose命令时，务必进入docker-compose.yml所在的目录，否则无法正常执行。

上传项目至服务器

路径：/opt/clims

配置docker-compose.yml
注意事项

宿主机下机数据路径务必与容器内部路径一致,请根据实际情况修改!

upload successful

version: ‘3’
services:
    db:
      container_name: mysql
      #image: mysql:5.6
      image: xx.local.genewiz.com/clims/mysql:5.6
      environment:
        MYSQL_ROOT_PASSWORD: xxxxxxxx
      ports:
        - “3306:3306”
      volumes:
        - ./mysql/data:/var/lib/mysql
 
    web:
        container_name: nginx
        #build: ./nginx
        image: xx.local.genewiz.com/clims/web:latest
        restart: always
        ports:
        - “80:80”
        volumes:
        - ./nginx/conf:/opt/nginx/conf
        - ./django/clims:/opt/clims-status
        depends_on:
        - status
 
    status:
        container_name: django
        #build: ./django
        image: xx.local.genewiz.com/clims/status:latest
        restart: always
        # command: uwsgi  –emperor uwsgi.ini
        command: uwsgi –ini /opt/clims/uwsgi/uwsgi.ini
        ports:
         - “8001:8001”
        volumes:
        - ./django/clims:/opt/clims
        depends_on:
         - db
 
    pipeline:
        container_name: pipeline
        #build: ./pipeline
        image: xx.local.genewiz.com/clims/pipeline:latest
        restart: always
        volumes:
        - ./pipeline/internal:/opt/pipeline/internal
        - ./pipeline/external:/opt/pipeline/external
        - ./pipeline/cron:/var/spool/cron
        - /sanger-pipeline/clims-in-uat:/sanger-pipeline/clims-in-uat
        - /sanger-pipeline/clims-out-uat:/sanger-pipeline/clims-out-uat
        depends_on:
         - db
         - status
         - web
 
volumes:
    data:
    clims:
    conf:

注意事项

此时构建镜像直接从本地仓库获取，而不是本地编译Dockerfile!



开始构建
docker-compose up -d
容器启停
容器停止
docker-compose stop
容器启动
docker-compose start
容器重启
docker-compose restart
容器重新构建
docker-compose up -d
Pipeline使用注意事项
Json配置文件说明
{
  "INPUT_ROOT_DIRECTORY": "/sanger-pipeline/clims-in-uat", //本地下机路径
  "OUTPUT_ROOT_DIRECTORY": "/sanger-pipeline/clims-out-uat", //本地下机路径
  "INPUT_SMB_DIRECTORY": "\\\\10.105.23.227\\seqdata-in", //相对应Samba路径
  "OUTPUT_SMB_DIRECTORY": "\\\\10.105.23.227\\seqdata", //相对应Samba路径
  "PEAKTRACE_SERVER":"10.105.21.181", //Peaktrace所在服务器IP
  "PEAKTRACE_USER":"root", //Peaktrace服务器账号
  "PEAKTRACE_PASSWORD":"xxxxxxxx", //Peaktrace服务器账号密码
  "PEAKTRACE_EXEC_ARGS": "-al no -c -10 -n 0 -xn yes", //Peaktrace参数
  "PEAKTRACE_UNITS":"140000", //告警阈值
  "PEAKTRACE_DAYS":"700", //告警阈值
  "MYSQL_HOST":"db", //容器DB主机
  "MYSQL_USER":"root", //容器DB账号
  "MYSQL_PASSWORD":"xxxxxxxx", //容器DB密码
  "MYSQL_DB":"pipelinedb", //数据库
  "STATUS_URL":"http://10.105.21.181/" // 详情页面地址
}
定时任务

默认均关闭，需要手动打开,请合理设置时间间隔！

# Internal Pipeline
#* * * * * python /opt/pipeline/internal/climspipe.py &>> /tmp/pipeline-internal.log
 
# External Pipeline
#* * * * * python /opt/pipeline/external/climspipe.py &>> /tmp/pipeline-external.log
 
# PeakTrace License Monitor
#* * * * * python /opt/pipeline/internal/pt_license_monitor.py

生产环境，已通过Peaktrace处理过的目录，请提前加入数据库中，否则会被重新处理，请慎重操作！

Nginx 域名配置（可选）

Nginx默认监听所有IP地址，容器部署完成后，直接访问宿主机IP地址即可！

如需配置域名，请替换如下域名，配置相应DNS记录,重启容器即可生效

upload successful

其他问题
docker WARNING: IPv4 forwarding is disabled? 
vim /etc/sysctl.conf or vim /usr/lib/sysctl.d/00-system.conf
 
添加如下代码：
net.ipv4.ip_forward=1
 
查看
sysctl -p
Demo

upload successful
upload successful
upload successful

0%