Multi-Agent Integration Documentation

Overview

The Multi-Agent integration connects to an external API hosted by Prosus that implements the Agent-to-Agent (A2A) protocol. This agent enables food ordering and restaurant search through iFood's Ailo agent, providing specialized capabilities for Brazilian food delivery services.

Unlike the Native Agent which runs client-side, the Multi-Agent communicates with a remote REST API service that handles the complex orchestration of multiple agents and integrations.

Architecture

High-Level Architecture

Key Characteristics

External API: Connects to Prosus-hosted API endpoint
A2A Protocol: Uses Agent-to-Agent protocol for inter-agent communication
iFood Integration: Specialized for iFood food ordering and restaurant search
Streaming Support: Supports both streaming (SSE) and non-streaming responses
Session Management: Maintains session state across conversations
Authentication: Uses iFood OAuth tokens for authenticated requests

API Endpoints

The Multi-Agent uses predefined endpoints hosted by Prosus:

Base URLs

Non-Streaming: https://6hm4cph5iu.eu-west-1.awsapprunner.com/chat/non-stream
Streaming: https://6hm4cph5iu.eu-west-1.awsapprunner.com/chat/stream

Note: These URLs are hardcoded and cannot be modified by users. They are defined in lib/database/repositories/agent-config.repository.ts.

Request Format

HTTP Method

POST

Headers

Header	Description	Required
`Content-Type`	`application/json`	Yes
`x-api-key`	Internal API key for authentication	Yes
`x-prosusai-ifood-access-token`	iFood OAuth access token	Conditional*
`x-prosusai-ifood-account-id`	iFood account ID	Conditional*
`x-prosusai-user-first-name`	User's first name	Optional
`x-prosusai-user-last-name`	User's last name	Optional
`x-prosusai-user-email`	User's email address	Optional

* Required when iFood authentication is available and needed for the request.

Request Body

typescript

{
  message: string;           // User's message content
  history: HistoryMessage[]; // Chat history
  mode: "super_tool";       // Always "super_tool" for multi-agent
  session_id?: string;      // Optional: Session ID for conversation continuity
}

History Message Format

typescript

interface HistoryMessage {
  role: "user" | "assistant";
  content: string;
}

Example Request (Non-Streaming)

bash

curl -X POST https://6hm4cph5iu.eu-west-1.awsapprunner.com/chat/non-stream \
  -H "Content-Type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "x-prosusai-ifood-access-token: ifood_token_here" \
  -H "x-prosusai-ifood-account-id: account_id_here" \
  -H "x-prosusai-user-first-name: John" \
  -H "x-prosusai-user-last-name: Doe" \
  -H "x-prosusai-user-email: john.doe@example.com" \
  -d '{
    "message": "Find me pizza restaurants near me",
    "history": [
      {
        "role": "user",
        "content": "Hello"
      },
      {
        "role": "assistant",
        "content": "Hi! How can I help you today?"
      }
    ],
    "mode": "super_tool",
    "session_id": "abc123-session-id"
  }'

Example Request (Streaming)

bash

curl -X POST https://6hm4cph5iu.eu-west-1.awsapprunner.com/chat/stream \
  -H "Content-Type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "x-prosusai-ifood-access-token: ifood_token_here" \
  -H "x-prosusai-ifood-account-id: account_id_here" \
  -d '{
    "message": "Order me a margherita pizza",
    "history": [],
    "mode": "super_tool"
  }'

Response Format

Non-Streaming Response

Returns a single JSON object:

typescript

{
  content: string;              // Agent's response text
  products?: Product[];         // Optional: Product items (for other integrations)
  ifoodSearchItems?: IfoodSearchItem[]; // Optional: iFood search results
  session_id?: string;          // Optional: Session ID for future requests
  type?: "search_items" | "text"; // Response type
}

iFood Search Items Format

typescript

interface IfoodSearchItem {
  // iFood-specific item structure
  // Contains restaurant, menu items, prices, etc.
}

Streaming Response (SSE)

Returns Server-Sent Events (SSE) format:

data: {"content": "I", "type": "text"}
data: {"content": " found", "type": "text"}
data: {"content": " some", "type": "text"}
data: {"content": " great", "type": "text"}
data: {"content": " options", "type": "text"}
data: {"content": "!", "type": "text"}
data: {"type": "search_items", "message": "Here are the restaurants:", "data": {"search_items": [...]}}
data: {"session_id": "abc123-session-id"}

Each line is a JSON object prefixed with data: .

SSE Event Types

Content Chunks: {"content": "text chunk", "type": "text"}
Search Items: {"type": "search_items", "message": "...", "data": {"search_items": [...]}}
Session ID: {"session_id": "session-id-string"}

Communication Modes

The Multi-Agent supports two communication modes:

1. Streaming Mode

Endpoint: /chat/stream

Behavior:

Returns Server-Sent Events (SSE) format
Content is streamed incrementally
Provides real-time feedback to users
Better UX for longer responses

Implementation: hooks/use-chat-custom-agent.ts - handleCustomAgentStreamingResponse()

2. Non-Streaming Mode

Endpoint: /chat/non-stream

Behavior:

Returns complete response in single JSON object
Simpler implementation
Useful when streaming is disabled in settings

Implementation: hooks/use-chat-custom-agent.ts - handleCustomAgentNonStreamingResponse()

Note: Polling mode is not supported for Multi-Agent.

Session Management

Session ID

The Multi-Agent API can return a session_id in responses to maintain conversation state:

Storage:

Stored in conversation record in local database
Field: session_id

Lifecycle:

First request → API may return new session_id
Subsequent requests → Include session_id in request body
Session persists until conversation is deleted

Implementation:

typescript

// Callback to save session_id when received
const handleSessionIdReceived = async (newSessionId: string) => {
  setSessionId(newSessionId);
  await conversationsDb.updateSessionId(convId, newSessionId);
};

// Include in request if available
body: JSON.stringify({
  message: content,
  history: history,
  mode: "super_tool",
  ...(sessionId && { session_id: sessionId }),
})

iFood Authentication

The Multi-Agent requires iFood OAuth tokens for authenticated requests:

Token Management

Tokens are managed by the iFood Auth Store
Automatically refreshed before API calls if needed
Stored securely in the app

Headers

When iFood authentication is available, these headers are automatically included:

x-prosusai-ifood-access-token: OAuth access token
x-prosusai-ifood-account-id: User's iFood account ID

Implementation: hooks/use-chat.ts - Automatically refreshes token before external agent calls.

Integration Flow

Code Examples

Example 1: Basic Request Flow

typescript

// In useChat hook
const sendMessage = async (content: string) => {
  // ... create assistant message
  
  // Get agent URLs
  const agentUrls = getAgentUrls(
    agentType,
    customAgentApiUrl,
    customAgentApiUrlStream,
    customAgentPollingUrl
  );
  
  if (agentType === 'multi-agent' && agentUrls) {
    // Refresh iFood token if needed
    await useIfoodAuthStore.getState().refreshTokenIfNeeded();
    
    // Get iFood auth
    const ifoodAuth = {
      accessToken: useIfoodAuthStore.getState().accessToken,
      accountId: useIfoodAuthStore.getState().accountId,
    };
    
    // Handle session ID callback
    const handleSessionIdReceived = async (newSessionId: string) => {
      setSessionId(newSessionId);
      await conversationsDb.updateSessionId(convId, newSessionId);
    };
    
    // Call streaming or non-streaming handler
    if (useStreaming) {
      await handleCustomAgentStreamingResponse(
        content,
        agentUrls.streamUrl,
        abortSignal,
        assistantMessageId,
        convId,
        updateMessage,
        handleStreamComplete,
        messages,
        ifoodAuth,
        sessionId,
        handleSessionIdReceived
      );
    } else {
      await handleCustomAgentNonStreamingResponse(
        content,
        agentUrls.apiUrl,
        abortSignal,
        assistantMessageId,
        convId,
        updateMessage,
        messages,
        ifoodAuth,
        sessionId,
        handleSessionIdReceived
      );
    }
  }
};

Example 2: Streaming Response Processing

typescript

// Inside handleCustomAgentStreamingResponse
const response = await fetch(apiUrl, {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "x-api-key": process.env.EXPO_PUBLIC_INTERNAL_API_KEY || "",
    ...(ifoodAuth?.accessToken && {
      "x-prosusai-ifood-access-token": ifoodAuth.accessToken,
    }),
    ...(ifoodAuth?.accountId && {
      "x-prosusai-ifood-account-id": ifoodAuth.accountId,
    }),
    ...(userFirstName && { "x-prosusai-user-first-name": userFirstName }),
    ...(userLastName && { "x-prosusai-user-last-name": userLastName }),
    ...(userEmail && { "x-prosusai-user-email": userEmail }),
  },
  body: JSON.stringify({
    message: content,
    history: history,
    mode: "super_tool",
    ...(sessionId && { session_id: sessionId }),
  }),
});

// Read SSE response
const fullText = await response.text();
const lines = fullText.split("\n");

let accumulatedContent = "";
let ifoodSearchItems: IfoodSearchItem[] | undefined;

for (const line of lines) {
  if (!line.trim() || !line.startsWith("data: ")) continue;
  
  const jsonString = line.substring(6); // Remove "data: " prefix
  const data = JSON.parse(jsonString);
  
  // Handle content chunks
  if (data.content) {
    accumulatedContent += data.content;
    updateMessage(assistantMessageId, {
      content: accumulatedContent,
      isStreaming: true,
    }, true, convId);
  }
  
  // Handle iFood search items
  if (data.type === "search_items") {
    ifoodSearchItems = data.data?.search_items || [];
    updateMessage(assistantMessageId, {
      content: data.message || "",
      ifoodSearchItems,
    }, true, convId);
  }
  
  // Handle session ID
  if (data.session_id && onSessionIdReceived) {
    onSessionIdReceived(data.session_id);
  }
}

Example 3: Non-Streaming Response Processing

typescript

// Inside handleCustomAgentNonStreamingResponse
const response = await fetch(apiUrl, {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "x-api-key": process.env.EXPO_PUBLIC_INTERNAL_API_KEY || "",
    ...(ifoodAuth?.accessToken && {
      "x-prosusai-ifood-access-token": ifoodAuth.accessToken,
    }),
    ...(ifoodAuth?.accountId && {
      "x-prosusai-ifood-account-id": ifoodAuth.accountId,
    }),
  },
  body: JSON.stringify({
    message: content,
    history: history,
    mode: "super_tool",
    ...(sessionId && { session_id: sessionId }),
  }),
});

const data = await response.json();

// Handle session_id
if (data.session_id && onSessionIdReceived) {
  onSessionIdReceived(data.session_id);
}

// Handle response based on type
let finalContent: string;
let ifoodSearchItems: IfoodSearchItem[] | undefined;

if (data.type === "search_items") {
  finalContent = data.message || "";
  ifoodSearchItems = data.data?.search_items || [];
} else {
  finalContent = data.content || "";
}

// Update UI
updateMessage(assistantMessageId, {
  content: finalContent,
  isStreaming: false,
  ifoodSearchItems,
}, false);

Configuration

Agent Type Selection

The Multi-Agent is selected in the Agent Settings screen:

File: app/agent-settings.tsx in the main application

Description: "Connects to iFood agent (Ailo) via Agent-to-Agent protocol. Enables food ordering and restaurant search through iFood."

Environment Variables

Variable	Description	Source
`EXPO_PUBLIC_INTERNAL_API_KEY`	Internal API key for Prosus-hosted endpoints	From .env file
`EXPO_PUBLIC_IFOOD_CLIENT_ID`	iFood OAuth Client ID	From .env file
`EXPO_PUBLIC_IFOOD_CLIENT_SECRET`	iFood OAuth Client Secret	From .env file

Important: Never commit API keys to version control. All keys should be stored in .env.local (which is git-ignored).

URL Configuration

The Multi-Agent URLs are hardcoded and cannot be modified by users:

File: lib/database/repositories/agent-config.repository.ts

typescript

export const MULTI_AGENT_API_URL = 'https://6hm4cph5iu.eu-west-1.awsapprunner.com/chat/non-stream';
export const MULTI_AGENT_API_URL_STREAM = 'https://6hm4cph5iu.eu-west-1.awsapprunner.com/chat/stream';

Error Handling

HTTP Errors

4xx Errors: Client errors (bad request, unauthorized, etc.)
5xx Errors: Server errors (internal server error, service unavailable, etc.)

All errors are caught and displayed to the user with appropriate error messages.

Network Errors

Connection timeouts
Network failures
Abort signal support for cancellation

Implementation

typescript

try {
  const response = await fetch(apiUrl, { ... });
  
  if (!response.ok) {
    throw new Error(`API request failed: ${response.statusText}`);
  }
  
  // Process response
} catch (error) {
  console.error("Error fetching multi-agent response:", error);
  throw error;
}

Response Types

Text Response

Standard text response from the agent:

json

{
  "content": "I found some great pizza restaurants near you!",
  "type": "text"
}

Search Items Response

iFood search results with restaurant/menu items:

json

{
  "type": "search_items",
  "message": "Here are the restaurants I found:",
  "data": {
    "search_items": [
      {
        "id": "restaurant-123",
        "name": "Pizza Place",
        "rating": 4.5,
        "deliveryTime": "30-45 min",
        "menuItems": [...]
      }
    ]
  }
}

Visual Response Display

When the Multi-Agent returns iFood search items:

Visual Components: Restaurant cards appear below the agent's message
Agent Response: Should be brief (1-2 sentences) as visual cards show all details
Display Format: iFood-specific UI components render restaurant information, menu items, prices, ratings, etc.

Limitations

External Dependency: Requires Prosus-hosted API to be available
iFood Only: Currently specialized for iFood integration
No Polling: Does not support polling mode (only streaming and non-streaming)
Hardcoded URLs: Endpoints cannot be customized by users
Network Required: Requires internet connection for all requests

Future Enhancements

Potential improvements:

Support for additional agent integrations beyond iFood
Configurable endpoints (if needed)
Polling mode support
Enhanced error recovery
Request retry logic

Native Agent - Native Agent documentation
Browser Agent - Browser Agent documentation
Custom Agent - Custom Agent documentation
Agent Settings UI: app/agent-settings.tsx in the main application
Custom Agent Handler: hooks/use-chat-custom-agent.ts in the main application

Multi-Agent Integration Documentation ​

Overview ​

Architecture ​

High-Level Architecture ​

Key Characteristics ​

API Endpoints ​

Base URLs ​

Request Format ​

HTTP Method ​

Headers ​

Request Body ​

History Message Format ​

Example Request (Non-Streaming) ​

Example Request (Streaming) ​

Response Format ​

Non-Streaming Response ​

iFood Search Items Format ​

Streaming Response (SSE) ​

SSE Event Types ​

Communication Modes ​

1. Streaming Mode ​

2. Non-Streaming Mode ​

Session Management ​

Session ID ​

iFood Authentication ​

Token Management ​

Headers ​

Integration Flow ​

Code Examples ​

Example 1: Basic Request Flow ​

Example 2: Streaming Response Processing ​

Example 3: Non-Streaming Response Processing ​

Configuration ​

Agent Type Selection ​

Environment Variables ​

URL Configuration ​

Error Handling ​

HTTP Errors ​

Network Errors ​

Implementation ​

Response Types ​

Text Response ​

Search Items Response ​

Visual Response Display ​

Limitations ​

Future Enhancements ​

Related Documentation ​

Multi-Agent Integration Documentation

Overview

Architecture

High-Level Architecture

Key Characteristics

API Endpoints

Base URLs

Request Format

HTTP Method

Headers

Request Body

History Message Format

Example Request (Non-Streaming)

Example Request (Streaming)

Response Format

Non-Streaming Response

iFood Search Items Format

Streaming Response (SSE)

SSE Event Types

Communication Modes

1. Streaming Mode

2. Non-Streaming Mode

Session Management

Session ID

iFood Authentication

Token Management

Headers

Integration Flow

Code Examples

Example 1: Basic Request Flow

Example 2: Streaming Response Processing

Example 3: Non-Streaming Response Processing

Configuration

Agent Type Selection

Environment Variables

URL Configuration

Error Handling

HTTP Errors

Network Errors

Implementation

Response Types

Text Response

Search Items Response

Visual Response Display

Limitations

Future Enhancements

Related Documentation