Fix the Worker Privilege warning in rConfig V8 Core
After reading this page, I can clear the Worker Privilege warning on the rConfig V8 Core dashboard by moving Horizon and the scheduler off root and onto the web server user, confirm the fix with rconfig:check-worker-user, and recheck the dashboard straight away. It explains every line the check can print and the fix for each one.
When to use this
Section titled “When to use this”Use this page when the dashboard Health Status panel shows a Worker Privilege alert, when sudo php artisan rconfig:check-worker-user reports a FAIL or WARN, or when update.sh or the installer stops at its final check. It applies to bare metal installs only. In Docker the container runs Horizon and the scheduler itself, so the check is skipped.
Why must the workers not run as root?
Section titled “Why must the workers not run as root?”Horizon runs every queued job, including device connections and config downloads, and the scheduler starts it all through schedule:run. If either runs as root, a flaw in any job runs with full control of the server. Root-run workers also leave root-owned files in storage and bootstrap/cache, which the web server then cannot write, causing permission errors and blank pages.
From V8.2.19+, both run as the web server user (apache on RHEL, Rocky and Alma, www-data on Ubuntu):
- Horizon runs under Supervisor as the
rconfig-core-horizonprogram withuser=set to the web user. Its config is a root-owned0644file:/etc/supervisord.d/rconfig-core-horizon.inion RHEL family systems,/etc/supervisor/conf.d/rconfig-core-horizon.confon Ubuntu. - The scheduler runs from
/etc/cron.d/rconfig-core, a root-owned0644file whose line runsschedule:runas the web user.
Installs from before V8.2.19 usually ran both as root. Upgrading with update.sh moves them over.
What does the Worker Privilege check report?
Section titled “What does the Worker Privilege check report?”The check runs four tests. The dashboard and rconfig:check-worker-user read the same report, so they always agree.
| Check | What it looks at |
|---|---|
| Horizon | The running artisan horizon processes for this rConfig folder, and which user owns them. |
| Scheduler cron file | That /etc/cron.d/rconfig-core exists, is a regular root:root 0644 file, and runs this folder’s schedule:run as the web user. |
| Root crontab | That root’s crontab has no schedule:run line for this folder, and that the scheduler has not run as root in the last 15 minutes. |
| Supervisor config | That the rconfig-core-horizon config exists, is a regular root:root 0644 file with user= set to the web user, and that no old horizon_supervisor file still starts this install. |
Only things that belong to this rConfig folder count. rConfig Pro’s Horizon, cron line or Supervisor file on the same server is never mistaken for Core’s.
The report sums up the server in one line:
| Overall | Meaning |
|---|---|
All good |
Nothing fails. The dashboard shows no alert. |
Still running as root |
At least one part runs as root and nothing yet runs as the web user. Typical of an install upgraded without update.sh. |
Mixed: some parts run as the web user and some as root |
The move is half done, for example a new Supervisor config while an old root crontab line still runs the scheduler. |
Not set up |
Horizon is not running and neither the cron file nor the Supervisor config exists. |
Misconfigured |
Something fails without pointing at root, for example a config file with the wrong owner or mode. |
On the dashboard, the alert shows the overall line followed by every problem found, and ends with the fix to run.
Prerequisites
Section titled “Prerequisites”- Shell access to the rConfig server with
sudo. - rConfig V8 Core V8.2.19+. Older releases have no
update.shorrconfig:check-worker-user. See How to update rConfig V8 Core. - The rConfig folder path. The examples use
/var/www/html/rconfig. - A short maintenance window. The fix stops Horizon and puts the app in maintenance mode while it runs.
- If rConfig Pro runs on the same server, nothing extra:
update.shonly touches this install’s Supervisor program and cron line.
How do I fix it?
Section titled “How do I fix it?”-
Go to the rConfig folder.
Terminal window cd /var/www/html/rconfig -
Run the check as root, so it can read root’s crontab.
Terminal window sudo php artisan rconfig:check-worker-userAn install upgraded without
update.shtypically shows:Web user: apache (checked as root)FAIL Horizon: Horizon runs as root (PID 3346344). It must run as apache only.FAIL Scheduler cron file: /etc/cron.d/rconfig-core is missing, so the scheduler is not set up to run as apache.OK Root crontab: Root's cron table does not run this install's scheduler.FAIL Supervisor config: /etc/supervisord.d/rconfig-core-horizon.ini is missing, so Supervisor does not run Horizon for this install.Overall: Still running as rootFix: Run sudo ./update.sh from /var/www/html/rconfigEach line starts with OK, WARN, FAIL or SKIP. Find any line you do not understand under What does each message mean?.
-
Run
update.shwithout pulling new code.Terminal window sudo ./update.sh --no-pull--no-pullreruns every step on the code you already have. The script stops Horizon, writes a root-owned Supervisor config withuser=set to the web user, writes/etc/cron.d/rconfig-core, removes this install’sschedule:runline from root’s and the web user’s crontabs (leaving every other line alone), handsstorageandbootstrap/cacheback to the web user, starts Horizon again, and runsrconfig:check-worker-useras its last step. It exits non-zero if that check still fails.Add
--skip-redis-secureif you do not want it to set a Redis password in the same run. See Fix the Redis Auth warning. -
Confirm the fix.
Terminal window sudo php artisan rconfig:check-worker-userecho $?Every line should read OK (or SKIP), the overall line should be
All good, and the exit code0. Only a FAIL exits non-zero, so a WARN still needs reading. -
Confirm Horizon now runs as the web user.
Terminal window sudo supervisorctl status rconfig-core-horizonps -eo user=,pid=,args= | grep '[/]var/www/html/rconfig/artisan horizon$'The status is
RUNNINGand thepsline starts withapacheorwww-data, notroot. -
Recheck the dashboard now rather than waiting for the next five minute run. Run the health checks as the web server user:
Terminal window # RHEL, Rocky, Almasudo -u apache php artisan health:check# Ubuntusudo -u www-data php artisan health:checkRefresh the dashboard. The Worker Privilege alert is gone. If the only thing left is “The scheduler ran as root in the last 15 minutes”, wait 15 minutes and check again (see below).
What does each message mean?
Section titled “What does each message mean?”The tables give the quick fix for each message. To see a problem for yourself, or to fix it without a full update.sh run, see Check and fix each item by hand.
Horizon
Section titled “Horizon”| Message | Fix |
|---|---|
Horizon runs as apache (PID …). (OK) |
Nothing to do. |
Horizon runs as root (PID …). It must run as apache only. |
Run sudo ./update.sh --no-pull. If it still runs as root afterwards, an old Supervisor program with a custom name, or a manual php artisan horizon, started it: see 1. Horizon. |
Horizon runs as apache (PID …) and root (PID …). It must run as apache only. |
Two Horizons are running, usually the new program plus an old one. Run sudo ./update.sh --no-pull, which removes the old Supervisor file when it belongs to this install. Stop any Horizon you started by hand. |
Horizon is not running, so its user could not be checked. (WARN) |
Start it with sudo supervisorctl start rconfig-core-horizon. If supervisorctl already shows it RUNNING, /proc is probably mounted with hidepid, which hides other users’ processes from the check. Check the Supervisor program’s user= line instead. |
Scheduler cron file
Section titled “Scheduler cron file”| Message | Fix |
|---|---|
/etc/cron.d/rconfig-core is missing, so the scheduler is not set up to run as apache. |
Run sudo ./update.sh --no-pull. To write only this file, run sudo ./setup_cron.sh from the rConfig folder. |
/etc/cron.d/rconfig-core is a symlink. |
The file must be a regular file root owns, so the web user cannot change what cron runs. Remove the symlink and run sudo ./setup_cron.sh. |
… is owned by …, not root:root. or … has mode …, not 0644. |
Cron skips group or world writable files. Run sudo ./setup_cron.sh, or fix it by hand with sudo chown root:root /etc/cron.d/rconfig-core && sudo chmod 0644 /etc/cron.d/rconfig-core. |
… runs the scheduler as root, not apache. |
Someone edited the user field. Run sudo ./setup_cron.sh to rewrite it. |
… has no line that runs this install's scheduler. |
The file exists but points at another folder or was emptied. Run sudo ./setup_cron.sh. |
The correct file holds one line like this (with www-data on Ubuntu):
* * * * * apache cd /var/www/html/rconfig && php artisan schedule:run >> /dev/null 2>&1Root crontab
Section titled “Root crontab”| Message | Fix |
|---|---|
Root's cron table still runs this install's scheduler as root: … |
Run sudo ./update.sh --no-pull, which removes that line and keeps the rest of root’s crontab. |
The scheduler ran as root in the last 15 minutes, but root's cron table has no line for it. (WARN) |
Something else ran schedule:run or health:check as root in the last 15 minutes. Look for it with grep -rn 'schedule:run' /etc/crontab /etc/cron.d/. If you ran php artisan health:check or schedule:run by hand as root, this is the cause: it clears on its own 15 minutes after the last root run. |
Not running as root, so root's cron table was not read. (SKIP) |
Run the check with sudo to include it. The dashboard runs it as the web user, so it shows SKIP there unless the scheduler itself runs as root. |
Supervisor config
Section titled “Supervisor config”| Message | Fix |
|---|---|
/etc/supervisord.d/rconfig-core-horizon.ini is missing, so Supervisor does not run Horizon for this install. |
Run sudo ./update.sh --no-pull. On Ubuntu the path is /etc/supervisor/conf.d/rconfig-core-horizon.conf. |
… has no user= line, so Supervisor runs Horizon as root. or … sets user=root, not apache. |
Run sudo ./update.sh --no-pull to rewrite the file. Never set user=root by hand. |
… is a symlink. |
Supervisor runs as root, so it must not load a file the web user can change. Remove the symlink and run sudo ./update.sh --no-pull. |
Old Supervisor config … still runs this install, so Supervisor may start a second Horizon from it, as root. |
An old horizon_supervisor.ini or .conf still starts this folder’s Horizon. update.sh removes it once it confirms it belongs to this install. If it belongs to rConfig Pro, it is not counted here. |
Could not tell the OS family from /etc/os-release, so the Supervisor config was not checked. (WARN) |
The OS is not RHEL family or Debian family. Confirm by hand that the Horizon program sets user= to the web user. |
Check and fix each item by hand
Section titled “Check and fix each item by hand”sudo ./update.sh --no-pull fixes every item in one go, and is the supported route. Use this section when you want to see a problem for yourself first, when update.sh has not cleared it, or when you cannot run a full upgrade right now. Each item has a manual check, then a manual fix that leaves the server exactly as update.sh would.
The examples use the RHEL family: web user apache, Supervisor folder /etc/supervisord.d/ with .ini files, cron service crond. On Ubuntu use www-data, /etc/supervisor/conf.d/ with .conf files, and cron. Replace /var/www/html/rconfig with your rConfig folder.
When you have finished, run sudo php artisan rconfig:check-worker-user and confirm every line reads OK.
1. Horizon
Section titled “1. Horizon”Check it. List every Horizon master for this folder with its user, and what started it:
ps -eo user=,pid=,ppid=,args= | grep '[/]var/www/html/rconfig/artisan horizon$'sudo supervisorctl statusgrep -l "/var/www/html/rconfig/artisan horizon" /etc/supervisord.d/* 2>/dev/nullGood looks like this: one ps line owned by apache, rconfig-core-horizon as RUNNING in supervisorctl, and grep listing only rconfig-core-horizon.ini.
If a ps line is owned by root, find its parent with ps -o args= -p <PPID>:
- The parent is
supervisord. A Supervisor program starts it. Every filegreplists, other thanrconfig-core-horizon.ini, starts a second Horizon for this folder.update.shonly removes the oldhorizon_supervisor.ini(programhorizon), so a file you made under another name, for examplehorizon_supervisor_v8core.iniwith[program:v8core_horizon], stays until you remove it. - The parent is a shell or
systemd. Someone started Horizon by hand, or from a custom systemd unit.
Fix it. Stop and remove every other Horizon for this folder, then make sure rconfig-core-horizon runs (set it up first, in item 4, if it does not exist yet):
# For each old program found above (use its name from supervisorctl status)sudo supervisorctl stop v8core_horizon# Move its file out of the folder. Supervisor loads every *.ini there, whatever it is called.sudo mv /etc/supervisord.d/horizon_supervisor_v8core.ini /root/sudo supervisorctl rereadsudo supervisorctl updatesudo supervisorctl status rconfig-core-horizonFor a Horizon started by hand, sudo kill <PID>. For a systemd unit, sudo systemctl disable --now <unit>.
Before you move a file, check that its command= line runs this folder’s artisan. On a shared server, never move rConfig Pro’s horizon_supervisor.ini or Prism’s files.
Horizon running as root leaves files in storage that belong to root. Hand them back to the web user, changing only files that are not already theirs:
cd /var/www/html/rconfigsudo find storage bootstrap/cache \( ! -user apache -o ! -group apache \) -exec chown -h apache:apache {} +If rconfig-core-horizon will not stay up (BACKOFF or FATAL), read sudo tail -n 50 /var/www/html/rconfig/storage/logs/horizon.log. The usual causes are root-owned files in storage (run the find above), including a root-owned horizon.log left by an older root-run Horizon, which stops the web user opening its log, Redis refusing the password (see Fix the Redis Auth warning), or a broken .env. If supervisorctl shows RUNNING but the check says Horizon is not running, /proc is mounted with hidepid: run the check with sudo.
2. Scheduler cron file
Section titled “2. Scheduler cron file”Check it.
ls -l /etc/cron.d/rconfig-corecat /etc/cron.d/rconfig-coreGood looks like this: a regular file (the ls line starts with -, not l), -rw-r--r--. 1 root root, and exactly one line that runs this folder’s schedule:run as apache.
Fix it. Run sudo ./setup_cron.sh from the rConfig folder. It writes only this file and cleans the crontabs (item 3), and it checks everything before writing anything. To write the file by hand instead:
sudo rm -f /etc/cron.d/rconfig-core # removes a symlink or a bad copysudo tee /etc/cron.d/rconfig-core >/dev/null <<'EOF'# rConfig V8 Core scheduler. Written by setup_cron.sh, do not edit: it is# rewritten on every install and upgrade. Runs as the web user, never root.SHELL=/bin/shPATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin* * * * * apache cd /var/www/html/rconfig && php artisan schedule:run >> /dev/null 2>&1EOFsudo chown root:root /etc/cron.d/rconfig-coresudo chmod 0644 /etc/cron.d/rconfig-coresudo restorecon -F /etc/cron.d/rconfig-core # RHEL with SELinux onlyKeep the name exactly rconfig-core: Ubuntu’s cron ignores files with a dot in the name, and every cron skips files that are group or world writable. Never leave a backup copy in /etc/cron.d, because RHEL’s cron runs those too.
Confirm cron runs it. The file only works if cron is running:
sudo systemctl enable --now crondsudo grep rconfig /var/log/cron | tail -n 3 # Ubuntu: sudo journalctl -u cron --since "5 min ago"Within a minute you should see schedule:run run as apache. If crond is missing, install it with sudo dnf install cronie (Ubuntu: sudo apt install cron).
3. Root crontab
Section titled “3. Root crontab”Check it.
sudo crontab -u root -l | grep schedule:runGood looks like this: no line that mentions this rConfig folder. A line for another folder, such as rConfig Pro’s, is fine.
Fix it. Do item 2 first, so the scheduler keeps running. Then run sudo ./setup_cron.sh, which removes only this folder’s lines, keeps the rest and saves a copy to /root/crontab-root.rconfig-bak-<date>. To do it by hand:
sudo crontab -u root -l > /root/crontab-root.backupsudo crontab -u root -e # delete only the line that runs this folder's schedule:runCheck the web user’s crontab the same way (sudo crontab -u apache -l). A line there makes the scheduler run twice.
“The scheduler ran as root in the last 15 minutes”. Root’s crontab is clean, but something ran schedule:run or health:check as root. Look for it:
sudo grep -rn "schedule:run" /etc/crontab /etc/cron.d/ /var/spool/cron/ 2>/dev/nullsystemctl list-timers --all | grep -i -e rconfig -e artisanRemove any line that runs this folder’s scheduler outside /etc/cron.d/rconfig-core. If nothing turns up, the cause was a root run by hand (sudo php artisan schedule:run or health:check). Run those as the web user from now on (sudo -u apache php artisan …). The warning clears 15 minutes after the last root run.
4. Supervisor config
Section titled “4. Supervisor config”Check it.
ls -l /etc/supervisord.d/rconfig-core-horizon.inicat /etc/supervisord.d/rconfig-core-horizon.iniGood looks like this: a regular file, root root, mode -rw-r--r--, with [program:rconfig-core-horizon], user=apache, a command= that appends to storage/logs/horizon.log, and stdout_logfile=/dev/null.
The check confirms the file, its owner, its mode and user=. It does not confirm the program name or the log path, so look at those yourself:
- The program name must be
rconfig-core-horizon.update.shstops and starts Horizon by that name. A file renamed by hand that still says[program:v8core_horizon]passes the check, butupdate.shwill not stop that Horizon before it changes the code. - Horizon writes its own log, and supervisord writes none.
command=runs Horizon through/bin/sh, which appends tostorage/logs/horizon.logas the web user.stdout_logfileandstderr_logfilemust stay/dev/null: supervisord runs as root, so any log it writes understorageleaves a root-owned file in the app folder.
Fix it. Run sudo ./setup_supervisor.sh from the rConfig folder. It renders the file from horizon_supervisor.ini in the rConfig folder, fills in the web user and folder, and refuses to write user=root. To write it by hand, first stop any old program for this folder (item 1), then:
sudo rm -f /etc/supervisord.d/rconfig-core-horizon.ini # removes a symlink or a bad copysudo tee /etc/supervisord.d/rconfig-core-horizon.ini >/dev/null <<'EOF'[program:rconfig-core-horizon]process_name=%(program_name)scommand=/bin/sh -c 'exec php /var/www/html/rconfig/artisan horizon >> /var/www/html/rconfig/storage/logs/horizon.log 2>&1'autostart=trueautorestart=trueuser=apachestdout_logfile=/dev/nullstdout_logfile_maxbytes=0stderr_logfile=/dev/nullstderr_logfile_maxbytes=0stopwaitsecs=3600EOFsudo chown root:root /etc/supervisord.d/rconfig-core-horizon.inisudo chmod 0644 /etc/supervisord.d/rconfig-core-horizon.inisudo supervisorctl rereadsudo supervisorctl updatesudo supervisorctl status rconfig-core-horizonNever symlink this file from the rConfig folder. Supervisor runs as root, so the web user must not be able to change what it starts. Writing the file by hand does not install the logrotate file for horizon.log, so run sudo ./setup_supervisor.sh once you are done.
If you renamed an old file in place and kept its old program name, stop the old program (sudo supervisorctl stop v8core_horizon) before you reread and update. If the old program left horizon.log owned by root, give it back to the web user: sudo chown apache:apache /var/www/html/rconfig/storage/logs/horizon.log.
If Supervisor is missing, install it: RHEL family sudo dnf install epel-release && sudo dnf install supervisor, Ubuntu sudo apt install supervisor, then sudo systemctl enable --now supervisord (supervisor on Ubuntu). If supervisorctl reread fails, another file in the folder has a syntax error, and the error names it.
When update.sh or the installer stops at the final check
Section titled “When update.sh or the installer stops at the final check”The upgrade or install itself has finished and the app is up. Only the final check failed. Read its output (for update.sh it is also in /var/log/rconfig-core-update-<date>.log), work through the items above for each FAIL line, and run the check again. On a fresh install the usual causes are a declined cron prompt or an installer that did not run as root (item 2), a skipped setup_supervisor.sh (item 4), or Horizon still restarting after the installer cleared its caches (wait a few seconds and check again).
How do I check from scripts or monitoring?
Section titled “How do I check from scripts or monitoring?”sudo php artisan rconfig:check-worker-user --jsonThe JSON holds state (all_good, still_root, mixed, not_set_up, misconfigured or docker), status (ok, warn, fail or skipped), the web_user, and one entry per check with its status and message. The exit code is 1 only when a check fails. WARN, SKIP and Docker exit 0.
What’s next
Section titled “What’s next”- Fix the Redis Auth warning, the other security check on the dashboard health panel
- Update rConfig V8 Core with update.sh and see every option it takes
- Manage the Horizon queue once it runs as the web user