Rating

When to Use

• Collecting quality scores for AI-generated summaries, responses, or decisions
• Post-task satisfaction ratings after completing a workflow step
• Feedback on AI recommendations before a user acts on them
• Performance review ratings submitted by a manager after a 1:1
• Quality scores for training data generation pipelines

Live Mockup

Request Payload Schema

interface RatingPayload {
  /** What is being rated (shown as the rating subject) */
  subject: string;

  /** Custom prompt shown above the star selector */
  prompt?: string;

  /** Whether to show the optional comment field (default: true) */
  allowComment?: boolean;

  /** Whether skipping (submitting without rating) is allowed (default: false) */
  allowSkip?: boolean;
}

Full Request Example

const response = await sendInteraction({
  type: 'rating',
  targetUserId: currentUserId,
  title: 'Rate this AI-generated summary',
  payload: {
    subject: `Contract summary for Agreement #${agreementId}`,
    prompt:  'How accurate and useful was the AI-generated summary?',
    allowComment: true,
    allowSkip: false   // must rate before proceeding
  },
  timeoutMs: 86_400_000,
  priority: 'low'
});

if (response.outcome === 'rated') {
  await feedbackService.recordRating({
    entityId:   agreementId,
    entityType: 'contract-summary',
    rating:     response.data.rating,
    comment:    response.data.comment,
    ratedBy:    response.respondedBy
  });
}

Response Schema

interface RatingResponse {
  interactionId: string;
  respondedBy:   string;
  outcome:       'rated' | 'skipped';
  data: {
    /** Star rating: 1 (poor) to 5 (excellent). Null if outcome is 'skipped'. */
    rating?: 1 | 2 | 3 | 4 | 5;

    /** Optional comment from the user */
    comment?: string;
  };
  timestamp: string;
}

Response Example

{
  "interactionId": "int_01HXRATING22Z",
  "respondedBy":   "usr_alice",
  "outcome":       "rated",
  "data": {
    "rating":  4,
    "comment": "Accurate overall but missed a key liability clause in section 7."
  },
  "timestamp": "2026-05-25T15:44:21Z"
}

Validation Rules

RLHF Use Case

The rating type is a natural fit for Reinforcement Learning from Human Feedback (RLHF) pipelines. Ratings are collected in the normal interaction inbox flow, with full audit trail (who rated, when, what rating, comment). These records can be exported from IInteractionAuditStore and fed directly into an RLHF training pipeline.

// Export rating interactions for RLHF training data
var ratings = await auditStore.QueryAsync(new AuditQuery
{
    Type     = "rating",
    Statuses = [InteractionStatus.Responded],
    CreatedAfter = DateTime.UtcNow.AddDays(-30)
});

var trainingData = ratings
    .Where(r => r.ResponseData?["rating"] != null)
    .Select(r => new RlhfSample
    {
        CorrelationId = r.CorrelationId,   // links to the AI output
        Rating        = (int)r.ResponseData["rating"],
        Comment       = r.ResponseData["comment"]?.ToString(),
        RatedBy       = r.RespondedBy,
        RatedAt       = r.RespondedAt!.Value
    });