May 12: Usage ingestion blocked by stuck Oban jobs
#May 12: Usage ingestion blocked by stuck Oban jobs (16:48UTC)
Usage ingestion fell behind and then stopped making progress when several background jobs hung indefinitely, eventually consuming all available worker concurrency for the ingestion queue. This was triggered by a bug in an Elixir decimal dependency where converting certain values (like 0.0) to an integer could loop forever, causing specific volume-usage receipt processing jobs to never finish. We fixed this by updating the dependency to a version containing the upstream bugfix, after which ingestion resumed and the backlog drained; once the queue cleared, the delayed ~18 hours of usage data was backfilled and reflected normally.