Parallel Agent Execution

When to Use Parallel Execution

Parallel execution is appropriate when:

• A single user request genuinely requires two or more independent domains (e.g., "What are my leave balance and my current expense report status?")
• The sub-questions are independent — one agent's result does not depend on another's
• Latency matters — sequential calls would be noticeably slower

Parallel execution should not be used when one specialist's output feeds into another's input — use sequential handoff in that case.

Parallel Execution API

public class ParallelAgentExecutor
{
    public async Task<ParallelAgentResult> ExecuteAsync(
        IReadOnlyList<ParallelAgentRequest> requests,
        ConversationComposite conversation,
        CancellationToken ct = default)
    {
        // Fan out: invoke all specialist agents concurrently
        var tasks = requests.Select(req => _agentExecutor.ExecuteAsync(
            agent: await _agentRepo.GetCompositeAsync(req.AgentId, conversation.TenantId),
            conversation: conversation,
            newMessage: req.FocusedPrompt,
            ct: ct));

        var results = await Task.WhenAll(tasks);

        // Fan in: merge results
        return new ParallelAgentResult
        {
            AgentResponses = requests.Zip(results).Select((pair, _) => new AgentResponse
            {
                AgentId = pair.First.AgentId,
                Content = pair.Second.Content,
                ToolCalls = pair.Second.ToolCalls
            }).ToList()
        };
    }
}

Response Merging

After parallel execution, the orchestrator merges results. Three merging strategies are supported:

// LLM Synthesis merge
var synthesisPrompt = $"""
The user asked: "{userMessage}"

Here are the specialist responses:
HR Agent: {hrResult.Content}
Finance Agent: {financeResult.Content}

Write a single, clear response that incorporates both answers.
""";

var merged = await _orchestratorLLM.CompleteAsync(
    new[] { new LLMMessage(Role.User, synthesisPrompt) },
    tools: null, options: orchestratorOptions);