---
title: Make the user_activity_audit writer resilient to pool saturation
type: task
status: planned
description: >-
  When the API connection pool saturates, the user_activity_audit insert times out and the activity
  log goes dark for the whole incident; isolate the write on its own pool with a bounded local spool
  so a saturated request path stops blinding the audit.
base_uri: user:task/league/make-user-activity-audit-writer-resilient-to-pool-saturation.md
created_at: '2026-09-22T06:58:38.997Z'
entity_id: 63266c29-1f42-4c81-88a8-061eabe557ff
owner_identity_uri: █████████████████████████
priority: Medium
public_read: true
relations:
  - relates [[████████████████████-████-████████-██████]]
tags:
  - user:tag/league-xo-football.md
  - user:tag/software-task.md
updated_at: '2026-09-22T06:58:38.997Z'
visibility_analyzed_at: '2026-11-05T23:49:41.772Z'
---

## Context

`log_user_activity` (`libs-server/log-user-activity.mjs:141`) writes each audit row with a single `database('user_activity_audit').insert(...)` on the module's MAIN request pool. It is deliberately best-effort: it runs from a response listener, can never block or change a response, and on failure emits ONE signal per process (`has_emitted_failure_signal`, `libs-server/log-user-activity.mjs:131`). It retries nothing. When the shared pool exhausts, the insert times out and the row is DROPPED — the activity log goes dark.

Measured 2026-09-19: a heavy import / data-view window saturated the knex pool (16:52–17:17 UTC, repeated `KnexTimeoutError: Knex: Timeout acquiring a connection` in the pm2 error logs, last at 17:17:21Z). During that window `log_user_activity` failed and the audit sank, as signal 130975's note says: "the activity log is dark until this is repaired". The knex error text about a missing `.transacting(trx)` is generic pool-exhaustion copy — this site is a single insert holding no transaction.

The failure is global pool saturation, so a retry on the same exhausted pool is not the repair; the repair is isolation and backpressure.

## Target

The audit writer should keep recording audit rows through a saturated request path. Pick the design you would build from scratch today (`user:guideline/design-toward-clean-end-state.md`):

- A dedicated, independently-sized knex pool for audit writes, threaded through the existing `database = db` parameter so `log_socket_activity` and `trx`-holding callers keep working unchanged.
- A bounded local spool: when even the dedicated pool cannot accept a write, hold the row in a bounded buffer and flush once the DB recovers, so "dark" decomposes into "delayed". Preserve the never-rejects contract and the single-signal-per-process gate, and re-arm the signal state after a flush.

## Plan

- [ ] Add a dedicated audit pool and route `log_user_activity`'s default `database` to it, leaving every explicit `database`/`trx` argument unchanged.
- [ ] Add a bounded local spool for rows the pool cannot accept; flush on recovery; keep the writer never-rejects and one-signal-per-process.
- [ ] Cover with a spec: saturate the shared request pool and assert the audit row still lands (or lands after flush) while responses remain unaffected.
- [ ] Re-arm `reset_failure_signal_state` path so a recovered writer can signal again on a later incident.
- [ ] Verify against the 09-19 failure mode on a staging/isolated pool before shipping the deploy.
