z4j-scheduler
v1.10.0 Apache-2.0z4j-scheduler
Engine-agnostic Python task scheduler. One service drives Celery, RQ, Dramatiq, Huey, arq, and taskiq.
z4j-scheduler is a Python task scheduler that drives any of the six engines z4j supports from one process. Schedules live in z4j's database; the dashboard creates, edits, pauses, resumes, and renames them without a daemon restart. The scheduler ticks once per second, dispatches due fires over gRPC to brain, and brain delivers the schedule.fire command to the agent, which enqueues the task on whichever broker the engine uses. The CLI imports Celery beat, django-celery-beat, rq-scheduler, APScheduler, and cron, with advisory exports for Celery, RQ, APScheduler, and cron.
- Latest
- v1.10.0
- Released
- 2026-08-28
- License
- Apache-2.0
Install
pip install z4j-schedulerCapabilities
- Engine-agnostic dispatch (Celery, RQ, Dramatiq, Huey, arq, taskiq)
- Live edit from dashboard, declarative config, or REST
- Per-schedule catch-up policy (skip / fire-one-missed / fire-all-missed)
- Explicit import targets plus advisory Celery, RQ, APScheduler, and cron exports
- Postgres advisory-lock leader for HA
- Tamper-evident HMAC-chained audit log of schedule edits
- Cron, interval, one-shot, and solar (sunrise/sunset) triggers
- Embedded mode (brain-supervised subprocess) or standalone process
Limitations
- Adds one process to your stack (or one subprocess in embedded mode)
- Brain becomes a hard dependency for schedule edits
- Not yet battle-tested at scale; operate with a fast-response posture and watch the CHANGELOG for recent load-test fixes
vs the existing Python schedulers
A factual capability matrix across the five most-used alternatives. All of them do their job well; the case for z4j-scheduler is the specific combination of engine-agnostic dispatch, live editing, and a tamper-evident audit chain in one service. If your stack runs one engine and one short schedule list, the existing native scheduler is probably the right call.
Engine & framework reach
| Capability | celery-beat | django-celery-beat | rq-scheduler | APScheduler 4 | system cron | z4j-scheduler |
|---|---|---|---|---|---|---|
| Engines supported | ~ Celery only | ~ Celery only | ~ RQ only | ~ in-process only | ~ shell exec | ✓ All 6 (Celery, RQ, Dramatiq, Huey, arq, taskiq) |
| Framework agnostic | ✓ | ✗ Django only | ✓ | ✓ | ✓ host OS | ✓ Django / Flask / FastAPI / bare |
| Multiple engines, one process | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ Celery + RQ + Huey side-by-side, one dashboard |
Operations & observability
| Capability | celery-beat | django-celery-beat | rq-scheduler | APScheduler 4 | system cron | z4j-scheduler |
|---|---|---|---|---|---|---|
| Edit live (no daemon restart) | ✗ | ~ Django admin only | ✗ | ~ persistent jobstore only | ✗ | ✓ Dashboard / declarative / REST |
| Built-in dashboard | ✗ | ~ Django admin (basic) | ✗ | ✗ | ✗ | ✓ Fire history, run-now, edit, audit log |
| Fire history per schedule | ✗ | ✗ | ✗ | ✗ | ~ syslog only | ✓ Buffered + acked + searchable |
| Audit log of schedule edits | ✗ | ~ 3rd-party django-auditlog | ✗ | ✗ | ✗ | ✓ HMAC-chained, tamper-evident |
| Manual fire-now button | ✗ | ✗ | ✗ | ~ API only | ✗ | ✓ Dashboard + REST |
| RBAC / project scoping | ✗ | ~ Django auth only | ✗ | ✗ | ~ UNIX permissions | ✓ Project + role-scoped (admin / operator / viewer) |
Reliability & catch-up
| Capability | celery-beat | django-celery-beat | rq-scheduler | APScheduler 4 | system cron | z4j-scheduler |
|---|---|---|---|---|---|---|
| HA / leader election | ✗ single instance only | ✗ single instance only | ✗ single instance only | ~ pluggable, manual | ✗ single host | ✓ Postgres advisory-lock leader, rolling-restart safe |
| Catch-up on outage | ~ all-or-nothing | ~ all-or-nothing | ~ default fire-all | ~ coalescing only | ✗ missed = lost | ✓ Per-schedule: skip / fire-one-missed / fire-all-missed |
| DST / IANA tz correctness | ✓ | ✓ | ~ UTC default | ✓ | ✓ | ✓ Validated at API; both fall-back instants remain distinct |
| Solar (sunrise / sunset / dusk) | ✓ celery.schedules.solar | ✓ | ✗ | ✗ | ✗ | ✓ Astronomical events with location |
Migration & lock-in
| Capability | celery-beat | django-celery-beat | rq-scheduler | APScheduler 4 | system cron | z4j-scheduler |
|---|---|---|---|---|---|---|
| Importer FROM other schedulers | - | - | - | - | - | ✓ All 6 native schedulers + crontab |
| Exporter TO other schedulers | - | - | - | - | - | ✓ No lock-in (round-trip pinned by tests) |
| Coexist with native scheduler | - | - | - | - | - | ✓ z4j-celerybeat coexistence adapter |
| Declarative-in-source schedules | ✓ beat_schedule dict | ✗ DB-only | ~ manual | ✓ Python | ✓ crontab file | ✓ z4j_scheduler.declarative reconciler |
Security
| Capability | celery-beat | django-celery-beat | rq-scheduler | APScheduler 4 | system cron | z4j-scheduler |
|---|---|---|---|---|---|---|
| Wire-protocol HMAC + replay protection | ~ broker-dependent | ~ broker-dependent | ✗ | ✗ | - | ✓ HMAC-SHA256 + per-session seq+nonce binding |
| Tamper-evident audit chain | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ Per-row HMAC, prev-hmac chain, DB-level UNIQUE |
| Secret rotation with a verification window | - | - | - | - | - | ~ Multi-key window for tokens and stored secrets; agents must be re-credentialed |
License & project shape
| Capability | celery-beat | django-celery-beat | rq-scheduler | APScheduler 4 | system cron | z4j-scheduler |
|---|---|---|---|---|---|---|
| License | ✓ BSD-3 | ✓ BSD-3 | ✓ MIT | ✓ MIT | ✓ BSD | ✓ Apache-2.0 |
| Process model | ✓ long-running daemon | ✓ daemon + DB | ✓ long-running daemon | ✓ in-process or daemon | ✓ init / systemd | ✓ Standalone OR brain-embedded subprocess |
| Idle CPU footprint | ✓ low | ✓ low | ✓ low | ✓ low | ✓ negligible | ✓ Negligible (1s tick, semaphore-bounded) |
Last verified 2026-08-24. We update this matrix when an upstream
scheduler ships a meaningful capability change. The data is in
sites/z4j-com/src/pages/schedulers/z4j-scheduler.astro
(top of frontmatter, COMPARISON array). PRs welcome.
Use cases the existing schedulers do not cover
Multi-engine shop, one source of truth
You run Celery for legacy services, RQ for a Django app, and arq for a FastAPI service. Each has its own schedule daemon today, with three different config formats and no shared audit. z4j-scheduler is one service that drives all three from one dashboard, with one audit log of every schedule edit.
Auditable schedule changes
An auditor wants to know who paused the nightly billing job last Tuesday and when. celery-beat does not keep a record. django-celery-beat keeps a partial one if you wired up django-auditlog. Every edit that goes through z4j (create, update, pause, delete) lands in an HMAC-chained log, and the database itself refuses a schedule change that arrives without a fresh revision and a matching change-log entry, so an edit cannot slip past its own record. A role writing those tables directly is outside that boundary; the database credentials deserve the same care as the audit key.
Live editing without restarts
Teams running celery-beat in Kubernetes know the dance: edit the static beat_schedule dict, restart the pod, hope nothing else breaks. z4j-scheduler picks up edits within one tick (one second by default), no restart. The schedule lives in Postgres, the dashboard writes to it, the scheduler reads from it.
Migration in or out, no lock-in
The CLI imports celery-beat static schedules, django-celery-beat rows, rq-scheduler, APScheduler jobstores, and system crontab. Advisory exports target Celery, RQ, APScheduler, and cron; the operator reviews and applies the generated file manually.
These are the shapes z4j-scheduler was designed around. If your stack matches one of them, this is the case it is meant to serve; if it does not, the per-engine adapter above is probably the better fit.
Three things, decoupled
Most Python schedulers fuse three concerns into one process: where the schedule lives, when it fires, and what runs the task. z4j-scheduler keeps them separate.
- Where the schedule lives: z4j's database. Edited from the dashboard, declarative config, or REST.
- When it fires: the z4j-scheduler service ticks once per second, dispatches due fires over gRPC to brain.
- What runs the task: your existing engine worker (Celery, RQ, arq, etc.) on its existing broker. z4j-scheduler does not run tasks itself.
The decoupling is what lets one z4j-scheduler instance manage schedules across a Celery project, an RQ project, and a Huey project from the same dashboard, with a single audit log of edits.
The parts that matter when it goes wrong
A scheduler is easy to write and hard to trust. These are the properties that decide whether you can leave it running.
- Retries of an accepted slot are de-duplicated. Every cadence fire carries an identity derived from the schedule and the exact time it was scheduled for. Retries of that slot therefore reach the brain with the same identity and collapse onto the same accepted command. This is not an exactly-once execution guarantee: a crash after the task is enqueued but before its result is known can leave the delivery outcome ambiguous, so scheduled tasks should be idempotent.
- Leadership mode is explicit.
The default
singlemode is for one scheduler process. PostgreSQLpostgresmode elects one global leader and keeps the other instances on standby. Onlypostgres_per_projectelects independently by project, allowing several instances to divide projects between them. - Missed fires are a decision, not an accident. When a schedule comes back from an outage with fires in the past, each schedule's catch-up policy decides whether to run none, one, or all of them. Nothing is silently swallowed, and nothing stampedes because it happened to be behind.
- You can see the change set before you apply it.
Run the importer with
--verifyagainst an existing celery-beat, django-celery-beat, rq-scheduler, APScheduler, or cron configuration and it prints what a real import would do to each schedule: insert, update, unchanged, or delete. That is a static diff of definitions, not a two-sided fire-time comparison, so it does not prove a cutover is safe. Check the imported schedules in the dashboard before you disable the source scheduler.
What a standalone scheduler cannot tell you
The advantage is not only the scheduler. It is that the thing firing your schedules and the thing watching your tasks are the same system, so a missed fire is an event with somewhere to go.
- A schedule that should have fired and did not becomes an alert. The control plane watches every enabled schedule independently of the scheduler process, so a dead or partitioned scheduler is detected by something that is not the scheduler. A standalone scheduler that stops cannot report that it stopped.
- A missed fire can act, not just notify. It feeds the automation rule engine like any other trigger, so it can page an operator or retry the work under the same governance, audit trail, and circuit breaker as everything else.
- Drift is measured, not assumed. Every fire records how far it landed from its scheduled time, so "the scheduler is getting slower" is a number with a graph rather than a suspicion.
- A manual fire records who did it. Cadence fires stay unattributed; a person pressing the button does not. Fire history is retained and queryable rather than trimmed to whatever the last log rotation kept.
Fit check
Greenfield, multiple engines, or multi-project: use z4j-scheduler. Define schedules from the dashboard or declarative config; one source of truth across the org.
Migrating from celery-beat / rq-scheduler / APScheduler: run the one-shot importer to copy existing schedules into z4j, verify in the dashboard, then disable the native scheduler. Or run both during a migration window using the coexistence adapter (z4j-celerybeat).
One engine, short schedule list, no team scaling concern: the existing native scheduler is probably fine. z4j-scheduler adds a process to your stack; that's only worth it if you'd benefit from the dashboard, audit chain, or cross-engine surface.
Two deploy modes
Embedded mode
For single-container homelab / small-deploy setups, z4j spawns z4j-scheduler as a supervised subprocess in its own lifespan and auto-mints loopback mTLS at boot. No extra ops surface.
Z4J_EMBEDDED_SCHEDULER=true
Z4J_SCHEDULER_GRPC_ENABLED=trueStandalone mode
For production, run z4j-scheduler as a separate process or container. Multiple instances elect a leader via Postgres advisory lock; only the leader ticks. Followers stay warm.
pip install z4j-scheduler
z4j-scheduler serve \
--brain-grpc-url brain.internal:7701 \
--brain-rest-url https://brain.internal \
--tls-cert /etc/z4j/scheduler.crt \
--tls-key /etc/z4j/scheduler.key \
--tls-ca /etc/z4j/ca.crtz4j-scheduler: FAQ
How is z4j-scheduler different from celery-beat?
celery-beat fires Celery tasks only and reads its schedule list from a Python dict, a Django database, or a redbeat backend that is fixed at start time. z4j-scheduler fires tasks on any of six engines (Celery, RQ, Dramatiq, Huey, arq, taskiq), keeps the schedule list in z4j's database, and lets you edit live from a dashboard or REST API without a daemon restart. The two are not direct substitutes; we ship z4j-celerybeat as a coexistence adapter so you can run them side by side during a migration window.
Is z4j-scheduler a Celery Beat alternative?
It can replace celery-beat for teams that want live editing from a dashboard, a tamper-evident audit log of schedule changes, multi-engine support, or HA leader election. The one-shot importer copies your existing celery-beat schedule (static dict or django-celery-beat rows) into z4j; you verify in the dashboard, then disable celery-beat. If your stack is pure Celery and a single daemon meets your needs, celery-beat is still a fine choice.
Does z4j-scheduler replace APScheduler?
Different niches. APScheduler runs in-process and is excellent for scheduling functions inside one Python application. z4j-scheduler is a separate companion service that dispatches to engine workers running elsewhere, with a dashboard, audit log, and cross-engine surface. We also ship z4j-apscheduler if you want to keep APScheduler and just surface its schedules in the z4j dashboard.
Is z4j-scheduler production-ready?
The protocol, audit chain, and HA leader-election paths have dedicated unit and integration coverage. It is not yet battle-tested at large scale. Operate with that in mind, especially for high-frequency schedule edits, and review the CHANGELOG before upgrading.
How does HA work?
Multiple z4j-scheduler instances connect to the same Postgres database and race for an advisory lock. Only the lock holder ticks; followers stay warm. After the database detects the leader connection loss, a follower can acquire the lock and start ticking. Takeover has no fixed response-time promise. The per-schedule catch-up policy decides what to do with missed slots, and HA does not make task execution exactly-once.
Can I run z4j-scheduler embedded inside z4j?
Yes. Embedded mode runs z4j-scheduler as a supervised subprocess inside z4j container, with auto-minted loopback mTLS at boot. This is the right pick for homelab and small-team deploys where adding another service is not worth the operational tax. Standalone mode is recommended once you scale past a single brain instance.
Will I get locked into z4j if I switch?
z4j-scheduler generates advisory source-shaped exports for Celery, RQ, APScheduler, and cron. If you decide to leave, review and apply the generated file in the target scheduler before disabling z4j-scheduler. Export does not write into deployment artifacts automatically.
What licence is z4j-scheduler under?
Apache 2.0. Importing z4j-scheduler into a commercial project does not affect your application's licensing. z4j (server + dashboard + API) is AGPL-3.0-or-later, isolated in its own process; the scheduler is separate.
Does z4j-scheduler handle DST and timezone correctness?
Yes. Cron schedules are validated against IANA time zones. During fall-back, an ambiguous wall-clock slot represents two distinct absolute instants and z4j fires once at each instant. Spring-forward behavior follows the cron calculation and the schedule's catch-up policy. Property tests sample common cron forms and seven representative zones over a five-year date range, with explicit New York transition checks.
