Debugging Stuck Interactions

Diagnosing: Is an Interaction Stuck?

An interaction is stuck (rather than merely slow) if all of these are true:

• It has been in-flight significantly longer than the P95 response time for its type
• The displayedAt field in the audit log is null (the UI never confirmed rendering)
• The target user is actively online (EdgeStream shows their session as connected)

Step-by-Step Debugging Process

Audit Log Query for Stuck Interactions

// Find interactions that have been in-flight for more than 2x their type's expected P95
public async Task<List<InteractionAuditEntry>> FindStuckInteractionsAsync(
    TimeSpan stuckThreshold,
    CancellationToken ct = default)
{
    return await _auditStore.QueryAsync(new AuditQuery
    {
        Statuses       = [InteractionStatus.Pending],
        CreatedBefore  = DateTime.UtcNow - stuckThreshold,
        // No displayedAt — never rendered by client
        DisplayedAt    = null
    }, ct);
}

// Example: find approvals stuck for more than 8 hours with no display
var stuck = await FindStuckInteractionsAsync(TimeSpan.FromHours(8), ct);
foreach (var entry in stuck)
{
    _logger.LogWarning(
        "Stuck interaction {Id} for user {User} — published {Age} ago, never displayed",
        entry.InteractionId,
        entry.TargetUserId,
        DateTime.UtcNow - entry.PublishedAt);
}

Manual Resolution API

Administrators can manually resolve a stuck interaction via the admin endpoint. This triggers the response pipeline as if the user had responded:

// POST /edge-interact/admin/resolve
[HttpPost("admin/resolve")]
[Authorize(Roles = "admin")]
public async Task<IActionResult> ResolveAsync(
    [FromBody] AdminResolveRequest req,
    CancellationToken ct)
{
    // Validate the interaction is still in pending state
    var entry = await _auditStore.GetByIdAsync(req.InteractionId, ct);
    if (entry is null)
        return NotFound();
    if (entry.Status != InteractionStatus.Pending)
        return Conflict($"Interaction is already in state: {entry.Status}");

    var response = new InteractionResponse
    {
        InteractionId = req.InteractionId,
        RespondedBy   = $"admin:{User.Identity!.Name}",
        Outcome       = req.Outcome,
        Data          = req.Data,
        Timestamp     = DateTime.UtcNow,
        AdminOverride = true
    };

    await _pipeline.SubmitResponseAsync(response, ct);
    return Ok(new { resolved = true, outcome = req.Outcome });
}

Manual Cancellation (Force Timeout)

To cancel an interaction without submitting a response (equivalent to forcing a timeout):

// POST /edge-interact/admin/cancel
[HttpPost("admin/cancel")]
[Authorize(Roles = "admin")]
public async Task<IActionResult> CancelAsync(
    [FromBody] AdminCancelRequest req,
    CancellationToken ct)
{
    await _pipeline.CancelAsync(req.InteractionId, reason: req.Reason, ct);
    return Ok();
}

Common Root Causes and Fixes

Grafana Query: Interactions Pending Over Threshold

// Count interactions in-flight for more than 1 hour
# Note: This requires the pipeline to tag in-flight metrics with publish timestamp
# Use the audit log query endpoint for fine-grained stuck detection instead.

# Coarse signal — growing in-flight gauge suggests accumulating stuck interactions:
interaction_in_flight > 50