pg_cron debugging guide

This is a general guide for debugging pg_cron. Below lists issues and how to debug them

Cannot create/edit/delete cron jobs#

Cron jobs can only be modified with the respective SQL functions:

Schedule: cron.schedule
Alter: cron.alter_job
Delete: cron.unschedule

If you are trying to make changes, use the cron functions. If the cron functions are inaccessible, contact Support

Cron Jobs are not running#

You should consider initiating a software upgrade in the General Settings if your Postgres version is below v15.6.1.122. Upgrading will give you access to pg_cron v1.6.4+, which has many bug fixes and auto-revive capabilities.

Debugging steps:#

Check to see if `pg_cron scheduler` is active#

pg_cron operates as the pg_cron scheduler process within Postgres. Use the below query to check if the worker is active

1
SELECT
2
  pid as process_id,
3
  usename as database_user,
4
  application_name,
5
  backend_start as when_process_began,
6
  wait_event_type,
7
  state,
8
  query,
9
  backend_type
10
FROM pg_stat_activity where application_name ilike 'pg_cron scheduler';

If the query does not return a row, the worker has died. To revive it, you must go to the General Settings and initiate a fast reboot:

Check the `cron.job_run_details` table for more information#

pg_cron creates logs in its own table cron.job_run_details. The below query checks for issues from the past 5 days :

1
SELECT *
2
FROM cron.job_run_details
3
WHERE
4
    (status <> 'succeeded' AND status <> 'running')
5
        AND
6
     start_time > NOW() - INTERVAL '5 days'
7
ORDER BY start_time DESC
8
LIMIT 10;

Respond to the errors exposed appropriately.

Long running jobs may show timeout errors. For jobs that are intended to execute for extended periods, consider wrapping their queries in functions with custom timeouts (guide).

Check if there are too many cron jobs running concurrently#

pg_cron supports up to 32 concurrent jobs, each using a database connection. If too many jobs are running simultaneously, space them out to prevent connection overload and job failure.

The below queries shows all pg_cron jobs:

1
-- All jobs
2
select schedule, jobname, command from cron.job;
3
4
-- Count jobs
5
select COUNT(*) from cron.job;

The below query can be used to find actively querying jobs:

1
SELECT
2
  pid as process_id,
3
  usename as database_user,
4
  application_name,
5
  backend_start as when_process_began,
6
  wait_event_type,
7
  state,
8
  query,
9
  backend_type
10
FROM pg_stat_activity where application_name ilike 'pg_cron';

You can view your concurrent peak connection usage throughout the day at the bottom of the Observability Dashboard

Check for database strain#

Unfortunately, excessive resource strain can slow down or disrupt jobs.

Go to the reports page (or Supabase Grafana if you have it setup), and check for signs of resource exhaustion. If it's clear your database is under pressure, consider upgrading your compute add-on or following the advice from one of the optimization guides:

It is important to make sure you are running the latest release of pg_cron (1.6.4) if you're noticing strain. It is the most robust.

Check the logs for more information#

Although pg_cron records errors in the cron.job_run_details table, in rare cases, more information can be found in the general Postgres logs. You can check the SQL Editor, with the query source set to Logs, for failure events with the following query

1
select
2
  timestamp,
3
  event_message,
4
  log_attributes['parsed.error_severity'] as error_severity,
5
  log_attributes['parsed.user_name'] as user_name,
6
  log_attributes['parsed.query'] as query,
7
  log_attributes['parsed.detail'] as detail,
8
  log_attributes['parsed.hint'] as hint,
9
  log_attributes['parsed.sql_state_code'] as sql_state_code,
10
  log_attributes['parsed.backend_type'] as backend_type,
11
  log_attributes['parsed.application_name'] as application_name
12
from logs
13
where
14
  source = 'postgres_logs'
15
  and log_attributes['parsed.error_severity'] in ('ERROR', 'FATAL', 'PANIC')
16
  and match(log_attributes['parsed.application_name'], 'pg_cron')
17
order by timestamp desc
18
limit 100;

If you're interested in modifying the query, there is an advanced guide for navigating the Postgres logs and a general-purpose one for applying filters.

Create custom logs within cron jobs#

If it's still not clear what is occurring you may be able to capture more logs by running the pg_cron query inside a database function:

1
create or replace function log_example()
2
returns void
3
language plpgsql
4
as $$
5
begin
6
7
     -- Logging start of function
8
     raise log 'logging start of cron function call: (%)', (select now());
9
10
    -- INSERT LOGIC HERE
11
12
     -- Logging end of function
13
     raise log 'logging end of cron function call: (%)', (select now());
14
15
    exception
16
     -- Handle exceptions here if needed
17
     when others then
18
         raise exception 'An error occurred in cron function <insert name here>. ERROR MESSAGE: %', sqlerrm;
19
20
end;
21
$$;

You can then search for your custom messages in the Logs Interface

Upgrading pg_cron version#

The current version of pg_cron on Supabase is 1.6.4. It comes with a few bug fixes. You should consider upgrading to Postgres v15.6.1.122+ in the General Settings to get the latest extension.

Contacting support and the maintainers#

Although Supabase includes the extension, it is maintained by Citus (a Microsoft subsidiary). You can contact Support for more help, but you should also consider creating an issue in the pg_cron repo.