Problem: Semaphore won´t work with git repos using SHA256 commit hashes (Postgres db as backend)

[pommetjehorlepiep]

Issue

Trying to run a simple bash task to create a file in the /tmp folder on the target host,

The task shows/keeps running in the UI, but Semaphore is actually terminated with a stacktrace.

time="2025-03-02T11:26:42+10:00" level=info msg="Task 3 added to queue"
time="2025-03-02T11:26:47+10:00" level=info msg="Set resource locker with TaskRunner 3"
time="2025-03-02T11:26:47+10:00" level=info msg="Task 3 removed from queue"
time="2025-03-02T11:26:47+10:00" level=panic msg="pq: value too long for type character varying(40)" error="Failed to update task commit"
time="2025-03-02T11:26:47+10:00" level=error msg="remove /tmp/semaphore/inventory_3: no such file or directory"
time="2025-03-02T11:26:47+10:00" level=info msg="Stopped running TaskRunner 3"
time="2025-03-02T11:26:47+10:00" level=info msg="Release resource locker with TaskRunner 3"
time="2025-03-02T11:26:47+10:00" level=panic msg="pq: value too long for type character varying(40)" error="Failed to update TaskRunner status"
panic: (*logrus.Entry) 0xc00020f1f0
panic: (*logrus.Entry) 0xc0002480e0

goroutine 161 [running]:
github.com/sirupsen/logrus.(*Entry).log(0xc000248070, 0x0, {0xc0006b8780, 0x31})
/home/runner/work/semaphore/semaphore/vendor/github.com/sirupsen/logrus/entry.go:260 +0x485
github.com/sirupsen/logrus.(*Entry).Log(0xc000248070, 0x0, {0xc00080a6b8?, 0xc00080a6c8?, 0x469085?})
/home/runner/work/semaphore/semaphore/vendor/github.com/sirupsen/logrus/entry.go:304 +0x48
github.com/sirupsen/logrus.(*Entry).Panic(...)
/home/runner/work/semaphore/semaphore/vendor/github.com/sirupsen/logrus/entry.go:342
github.com/semaphoreui/semaphore/util.LogPanicF({0x1b2d920, 0xc00046ea20}, 0xd1d111?)
/home/runner/work/semaphore/semaphore/util/errorLogging.go:39 +0x88
github.com/semaphoreui/semaphore/services/tasks.(*TaskRunner).panicOnError(0x3?, {0x1b2d920, 0xc00046ea20}, {0xd3ad36, 0x22})
/home/runner/work/semaphore/semaphore/services/tasks/TaskRunner_logging.go:135 +0x165
github.com/semaphoreui/semaphore/services/tasks.(*TaskRunner).saveStatus(0xc0001f2608)
/home/runner/work/semaphore/semaphore/services/tasks/TaskRunner.go:78 +0x65c
github.com/semaphoreui/semaphore/services/tasks.(*TaskRunner).run.func1()
/home/runner/work/semaphore/semaphore/services/tasks/TaskRunner.go:116 +0x196
panic({0xd0d4a0?, 0xc00020f1f0?})
/opt/hostedtoolcache/go/1.23.6/x64/src/runtime/panic.go:785 +0x132
github.com/sirupsen/logrus.(*Entry).log(0xc00020f180, 0x0, {0xc0001a9180, 0x31})
/home/runner/work/semaphore/semaphore/vendor/github.com/sirupsen/logrus/entry.go:260 +0x485
github.com/sirupsen/logrus.(*Entry).Log(0xc00020f180, 0x0, {0xc00080aea0?, 0xc00080aeb0?, 0x469085?})
/home/runner/work/semaphore/semaphore/vendor/github.com/sirupsen/logrus/entry.go:304 +0x48
github.com/sirupsen/logrus.(*Entry).Panic(...)
/home/runner/work/semaphore/semaphore/vendor/github.com/sirupsen/logrus/entry.go:342
github.com/semaphoreui/semaphore/util.LogPanicF({0x1b2d920, 0xc000238480}, 0xd1d111?)
/home/runner/work/semaphore/semaphore/util/errorLogging.go:39 +0x88
github.com/semaphoreui/semaphore/services/tasks.(*TaskRunner).panicOnError(0x3?, {0x1b2d920, 0xc000238480}, {0xd33546, 0x1c})
/home/runner/work/semaphore/semaphore/services/tasks/TaskRunner_logging.go:135 +0x165
github.com/semaphoreui/semaphore/services/tasks.(*TaskRunner).SetCommit(0xc0001f2608, {0xc0002200f0, 0x40}, {0xc000189728, 0x16})
/home/runner/work/semaphore/semaphore/services/tasks/TaskRunner_logging.go:72 +0x19e
github.com/semaphoreui/semaphore/services/tasks.(*LocalJob).SetCommit(...)
/home/runner/work/semaphore/semaphore/services/tasks/LocalJob.go:57
github.com/semaphoreui/semaphore/services/tasks.(*LocalJob).checkoutRepository(0xc0001f2c08)
/home/runner/work/semaphore/semaphore/services/tasks/LocalJob.go:642 +0x36b
github.com/semaphoreui/semaphore/services/tasks.(*LocalJob).prepareRun(0xc0001f2c08, {0x0, 0x0, 0x0}, {0xbd6820, 0x209df20})
/home/runner/work/semaphore/semaphore/services/tasks/LocalJob.go:553 +0x2ac
github.com/semaphoreui/semaphore/services/tasks.(*LocalJob).Run(0xc0001f2c08, {0xc000746640, 0x6}, 0x0, {0x0, 0x0})
/home/runner/work/semaphore/semaphore/services/tasks/LocalJob.go:470 +0x2c7
github.com/semaphoreui/semaphore/services/tasks.(*TaskRunner).run(0xc0001f2608)
/home/runner/work/semaphore/semaphore/services/tasks/TaskRunner.go:163 +0x699
created by github.com/semaphoreui/semaphore/services/tasks.(*TaskPool).Run in goroutine 7
/home/runner/work/semaphore/semaphore/services/tasks/TaskPool.go:186 +0x4b6

Impact

Web-Frontend (what users interact with)

Installation method

Binary

Database

Postgres

Browser

No response

Semaphore Version

2.12.14-685c726-1739570559

Ansible Version

ansible [core 2.18.3]
config file = None
configured module search path = ['/home/semaphore/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /home/semaphore/.local/lib/python3.12/site-packages/ansible
ansible collection location = /home/semaphore/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/bin/ansible
python version = 3.12.9 (main, Feb  7 2025, 17:39:29) [GCC 14.2.0] (/usr/bin/python3)
jinja version = 3.1.5
libyaml = True

Logs & errors

No response

Manual installation - system information

OS: Alpine 3.21

Configuration

No response

Additional information

No response

[pommetjehorlepiep]

Ok, did a bit of digging in the code and I worked out the problem.

create table project__schedule
(
    id               integer primary key autoincrement,
    template_id      int          not null,
    project_id       int          not null,
    cron_format      varchar(255) not null,
    repository_id    int          null,
    last_commit_hash varchar(40)  null,

    foreign key (`template_id`) references project__template(`id`) on delete cascade,
    foreign key (`project_id`) references project(`id`) on delete cascade,
    foreign key (`repository_id`) references project__repository(`id`)
)

The problem is with the last_commit_hash field which is defined as 40 characters.
40 chars is the length of a SHA1 commit hash...

The git repo I'm using is set to use SHA256, which translates in a hash of 64 hex characters.

[pommetjehorlepiep]

Verified that this is the actual problem by:

ALTER TABLE project__schedule ALTER COLUMN last_commit_hash TYPE VARCHAR(64);
ALTER TABLE task ALTER COLUMN commit_hash TYPE VARCHAR(64);

Original errors goes away + task gets executed without problem!