Practical GitLab 18 Fix — Complete English Guide!

Operations Dec 17, 2025

Case Study: Fixing Corrupted `project_daily_statistics` Table in GitLab 18.6.1

Symptoms: GitLab Web UI returns 502 errors, Git operations over SSH/HTTPS fail with error 500, and background jobs crash — all traced back to an invalid database schema after upgrading to GitLab 18.6.1.

🔍 Background

I was running a self-managed GitLab CE instance on CentOS 7.x. Since CentOS 7 reached end-of-life, my GitLab version was stuck at 17.7.7, preventing me from using newer features.

I decided to migrate to a supported OS. An initial attempt with Debian failed due to UID/GID issues, so I successfully moved to Rocky Linux 9.6. After the migration, I attempted to upgrade GitLab incrementally.

Upgrades went smoothly until I jumped from 18.5.x directly to 18.6.1. The upgrade appeared successful, but I soon discovered:

SSH-based Git operations worked fine.
HTTP/HTTPS Git operations returned 500 Internal Server Error.
The Web UI remained fully functional.

After extensive debugging, I found that upgrading step-by-step (18.5.x → 18.6.0 → 18.6.1) revealed the real issue: the upgrade to 18.6.1 failed due to a corrupted database table — specifically, project_daily_statistics.

🔍 Problem Summary

In GitLab ≥18.0, the project_daily_statistics table was redesigned to track only fetch activity (used for license compliance and usage reporting). However, during the upgrade to 18.6.1, the database migration was incomplete or skipped, leaving the table in a hybrid state:

It retained legacy columns from pre-18.0 versions (e.g., total_repository_size, commit_count, created_at, etc.).
It was missing the new required column: fetch_count.

This mismatch caused:

Rails (Puma) to crash intermittently or fail to start.
Git authentication logic to break for HTTP/HTTPS requests.
Sidekiq workers (e.g., Projects::DailyStatisticsWorker) to raise ActiveRecord::StatementInvalid exceptions.

✅ Correct Table Schema (GitLab 18.6.1)

The official structure for project_daily_statistics in GitLab ≥18.0 is:

Partitioned table "public.project_daily_statistics"
Column        | Type    | Nullable | Default
--------------+---------+----------+-------------------
id            | bigint  | not null | nextval(...)
project_id    | integer | not null |
fetch_count   | integer | not null | 0
date          | date    | not null |

Partition key: RANGE (date)
Primary key: (id, date)
Unique index: (project_id, date DESC)

⚠️ Note: All _size fields and timestamp columns (created_at, updated_at) were removed in v18.0.

🛠️ Step-by-Step Fix

⚠️ Warning: Always create a backup before making database changes!

sudo gitlab-backup create

1. Connect to the GitLab PostgreSQL database

sudo gitlab-psql

2. Drop obsolete columns

ALTER TABLE project_daily_statistics
  DROP COLUMN IF EXISTS total_repository_size,
  DROP COLUMN IF EXISTS commit_count,
  DROP COLUMN IF EXISTS storage_size,
  DROP COLUMN IF EXISTS lfs_objects_size,
  DROP COLUMN IF EXISTS build_artifacts_size,
  DROP COLUMN IF EXISTS pipeline_artifacts_size,
  DROP COLUMN IF EXISTS packages_size,
  DROP COLUMN IF EXISTS wiki_size,
  DROP COLUMN IF EXISTS snippets_size,
  DROP COLUMN IF EXISTS job_artifacts_size,
  DROP COLUMN IF EXISTS created_at,
  DROP COLUMN IF EXISTS updated_at;

3. Add the missing `fetch_count` column

ALTER TABLE project_daily_statistics
  ADD COLUMN fetch_count INTEGER NOT NULL DEFAULT 0;

4. Rebuild the primary key (must include partition key `date`)

-- Drop old index if it exists
DROP INDEX IF EXISTS index_project_daily_statistics_on_project_id_and_date;

-- Add composite primary key
ALTER TABLE project_daily_statistics
  ADD PRIMARY KEY (id, date);

5. Recreate the unique index

❗ PostgreSQL partitioned tables do not support CREATE INDEX CONCURRENTLY.

CREATE UNIQUE INDEX idx_p_project_daily_statistics_on_project_id_and_date
  ON project_daily_statistics (project_id, date DESC);

6. Restart GitLab

sudo gitlab-ctl restart

💡 Tip: On low-memory systems, Puma may take 1–3 minutes to start. Please wait before testing!

✅ Verification

Confirm the table structure:

\d+ project_daily_statistics;

Check service status:

sudo gitlab-ctl status

Test all access methods:

Web UI: https://your-gitlab.com
Git over SSH: ssh -T [email protected]
Git over HTTPS: git clone https://...

All should work without errors.

📌 Key Takeaways

Never manually alter GitLab database tables unless absolutely necessary. Prefer gitlab-rake db:migrate.
GitLab 18.0+ removed repository size tracking from daily statistics — old fields are obsolete.
PostgreSQL partitioned tables cannot use CONCURRENTLY when creating indexes.
Slow Puma startup after restart is normal on resource-constrained servers — be patient!

🔗 References

✨ Verified and fixed on GitLab 18.6.1 (Omnibus install).
Shared to help other admins avoid hours of debugging!