雖然一直以來我們團隊自建的 GitLab 都是用 docker-compose 來啟動,直接把 container mounting volume 備份起來是最乾淨快速的方法(記得壓縮與解壓時要保留原本的檔案權限),不過整個 GitLab 的資料夾其實蠻大的,而且查到的資料是不建議在 GitLab 的 PostgreSQL running 的情況下直接備份(可能會有資料庫不一致的情況),最後我還是使用了官方的備份與還原流程
官方的流程雖然比較複雜,不過比起備份整個 volume,速度還是比較快,而且不需要停止 GitLab。在實測備份與還原後,還原的過程也相當的順利,下面就來記錄備份與還原的步驟(主要是還原)
VM 環境
GitLab Version: 19.2
OS Version: Ubuntu 24.04 LTS
GitLab Docker-Compose
以下是我用來啟動 GitLab 的 Docker-Compose 內容,其中 SMTP 部分可視需要把註解取消,另外也加了一個 GitLab Runner:
# .env
GITLAB_HOME=/srv/gitlab
services:
gitlab:
image: gitlab/gitlab-ce:19.2.0-ce.0
container_name: gitlab
restart: always
env_file: .env
environment:
GITLAB_OMNIBUS_CONFIG: |
external_url "https://gitlab.mydomain.com"
prometheus['storage_retention'] = '15d'
### registry
registry['enable'] = false
### disable bundle nginx
nginx['enable'] = false
gitlab_workhorse['listen_network'] = "tcp"
gitlab_workhorse['listen_addr'] = "0.0.0.0:8181"
###
gitlab_rails['registry_enabled'] = false
gitlab_rails['cache_clearer_enable'] = true # Auto clear rails cache
#gitlab_workhorse['client_max_body_size'] = 1073741824
letsencrypt['enable'] = false
### gmail smtp
#gitlab_rails['gitlab_email_enabled'] = true
#gitlab_rails['gitlab_email_from'] = 'xxx@mydomain.cc'
#gitlab_rails['gitlab_email_display_name'] = 'xxx'
#gitlab_rails['smtp_enable'] = true
#gitlab_rails['smtp_address'] = "smtp.gmail.com"
#gitlab_rails['smtp_port'] = 587
#gitlab_rails['smtp_user_name'] = '${SMTP_USER_NAME}'
#gitlab_rails['smtp_password'] = '${SMTP_PASSWORD}'
#gitlab_rails['smtp_domain'] = "smtp.gmail.com"
#gitlab_rails['smtp_authentication'] = "login"
#gitlab_rails['smtp_enable_starttls_auto'] = true
#gitlab_rails['smtp_tls'] = false
#gitlab_rails['smtp_openssl_verify_mode'] = 'peer'
### prometheus
# enable prometheus metrics
gitlab_rails['prometheus_metrics_enabled'] = true
gitlab_rails['monitoring_whitelist'] = ['10.10.10.1/24', '127.0.0.1']
prometheus['listen_address'] = '0.0.0.0:9090'
prometheus['flags'] = {
'storage.tsdb.retention.time' => "7d",
'storage.tsdb.retention.size' => "2GB",
}
volumes:
- '${GITLAB_HOME}/config:/etc/gitlab'
- '${GITLAB_HOME}/logs:/var/log/gitlab'
- '${GITLAB_HOME}/data:/var/opt/gitlab'
logging:
driver: "json-file"
options:
max-size: "2047m"
max-file: "5"
ports:
- "9090:9090"
shm_size: '256m'
stop_grace_period: '60s'
networks:
- web
gitlab-runner:
image: gitlab/gitlab-runner:alpine
container_name: gitlab-runner
restart: unless-stopped
depends_on:
- gitlab
volumes:
# 給 runner 自己的設定檔(映到容器內 /etc/gitlab-runner)
- /srv/gitlab-runner/config:/etc/gitlab-runner
# 讓 runner 能在「這台 VM」上啟動 job/sidecar 容器(docker executor 需要)
- /var/run/docker.sock:/var/run/docker.sock
environment:
- TZ=Asia/Taipei
networks:
- web
networks:
web:
external: false
GitLab Backup Script
以下是我用來備份 GitLab 的 script 內容,雲端部分我是備份到 GCP 的 Cloud Storage:
#!/bin/bash
set -euo pipefail
GITLAB_HOME=/srv/gitlab
GCS_BUCKET=gs://xxx-gitlab-backup
STAGING_DIR=/tmp/gitlab-backup-staging
LOG_FILE=/var/log/gitlab-backup.log
log() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*" | tee -a "$LOG_FILE"; }
log "=== GitLab backup started ==="
# 1. 應用備份(repo、DB、uploads 等)
log "Running gitlab-backup create..."
docker exec gitlab gitlab-backup create SKIP=registry CRON=1
# 2. 取得最新備份檔
LATEST_TAR=$(ls -t "${GITLAB_HOME}/data/backups/"*_gitlab_backup.tar 2>/dev/null | head -1)
if [ -z "$LATEST_TAR" ]; then
log "ERROR: No backup file found"
exit 1
fi
# 從備份檔名取出版本號(例:19.2.0 → 19.2)
VERSION=$(basename "$LATEST_TAR" | grep -oP '\d+\.\d+\.\d+' | cut -d. -f1,2)
DATETIME=$(date +%Y%m%d-%H%M%S)
ARCHIVE_NAME="gitlab-v${VERSION}-backup-${DATETIME}.tgz"
log "Archive: $ARCHIVE_NAME"
# 3. 打包 backup tar + secrets.json 成一個 tgz
mkdir -p "$STAGING_DIR"
TGZ="${STAGING_DIR}/${ARCHIVE_NAME}"
tar -czf "$TGZ" \
-C "${GITLAB_HOME}/data/backups" "$(basename $LATEST_TAR)" \
-C "${GITLAB_HOME}/config" gitlab-secrets.json
# 4. 上傳到 GCS
log "Uploading to GCS..."
gcloud storage cp "$TGZ" "${GCS_BUCKET}/"
log "Upload complete"
# 5. 清除本機暫存與原始備份
rm -f "$TGZ"
rm -f "$LATEST_TAR"
rmdir --ignore-fail-on-non-empty "$STAGING_DIR"
log "=== GitLab backup finished ==="
上面的 script 最重要其實就是執行
docker exec gitlab gitlab-backup create SKIP=registry CRON=1
這個官方的備份指令,加上把 secrets.json 也一起打包進去(不然無法還原加密的 Variables)。最後再把 $ARCHIVE_NAME 改個自己喜歡的格式就可以了,上傳的空間可以自己再選別的,我的這台 VM 因為有設 SA,所以可以直接上傳到 GCS
GitLab Restore Steps
因為我是用原本的 GitLab VM 來還原,所以首先要把 GitLab 關閉並刪除原本的資料來模擬乾淨的還原
docker compose down gitlab
sudo rm -rf "${GITLAB_HOME}/data"
sudo rm -rf "${GITLAB_HOME}/config"
sudo rm -rf "${GITLAB_HOME}/logs"
接下來把 GitLab 啟動
docker compose up -d gitlab
啟動之後你可以在 browser 看到一個乾淨的 GitLab,接著把備份檔的 tar 與 secrets.json 解壓縮到 GitLab 的資料夾中
sudo cp gitlab-secrets.json "${GITLAB_HOME}/config/"
sudo cp ${GITLAB_BACKUP_FILE}.tar "${GITLAB_HOME}/data/backups/"
接下來要為 ${GITLAB_BACKUP_FILE} 設定正確的 owner,有兩種設法:
- 進去容器裡修改
docker exec -it gitlab bash
chown gitlab:gitlab /var/opt/gitlab/backups/${GITLAB_BACKUP_FILE} && exit
``L
2. 直接在主機上修改
```sh
sudo chown 998:998 "${GITLAB_HOME}/data/backups/${GITLAB_BACKUP_FILE}"
接著參考官方的範例,執行下面的指令
docker exec -it gitlab gitlab-ctl stop pumat
docker exec -it gitlab gitlab-ctl stop sidekiq
檢查上面兩個 process 是否都已經停止
docker exec -it gitlab gitlab-ctl status
執行還原,請記得將 ${GITLAB_BACKUP_FILE} 替換成你要還原的備份檔名
docker exec -it gitlab gitlab-backup restore BACKUP=${GITLAB_BACKUP_FILE}
重啟 GitLab
docker compose restart gitlab
檢查還原是否成功
docker exec -it gitlab gitlab-rake gitlab:check SANITIZE=true
沒問題的話到這裡還原就完成了,另外官方有提到一個 reconfigure 的步驟,因為我們的 config.rb 內容是寫在 docker-compose.yml 裡,所以不需要這個步驟
參考資料: