Phidata Integration

Use Portkey AI Gateway with Phidata for AI assistants with memory and knowledge

Integrate Portkey with Phidata to build production-ready AI assistants with access to 250+ LLMs, automatic fallbacks, and complete observability.

Overview

Portkey enhances Phidata applications with:

• Multi-Provider Support: Connect to 250+ LLMs for your AI assistants
• Reliability: Automatic fallbacks and retries for assistant interactions
• Observability: Full logging and tracing for assistant conversations
• Performance: Smart caching to improve response times
• Cost Optimization: Track and reduce token usage

Installation

pip install portkey-ai phidata

Quick Start

Phidata integrates with Portkey through OpenAI-compatible configuration:

1. Import Libraries

   from phi.assistant import Assistant
   from phi.llm.openai import OpenAIChat
   from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders
   
2. Configure Portkey

   portkey_headers = createHeaders(
       api_key="your-portkey-api-key",
       provider="openai"
   )
   
3. Create LLM with Portkey

   llm = OpenAIChat(
       model="gpt-4",
       api_key="your-openai-api-key",
       base_url=PORTKEY_GATEWAY_URL,
       default_headers=portkey_headers
   )
   
4. Create Assistant

   assistant = Assistant(
       llm=llm,
       description="You are a helpful AI assistant"
   )
   
5. Use the Assistant

   assistant.print_response("What is quantum computing?")
   

Complete Assistant Example

Build a comprehensive AI assistant:

from phi.assistant import Assistant
from phi.llm.openai import OpenAIChat
from phi.tools.duckduckgo import DuckDuckGo
from phi.tools.wikipedia import WikipediaTools
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders

# Configure Portkey
portkey_headers = createHeaders(
    api_key="your-portkey-api-key",
    provider="openai",
    metadata={
        "environment": "production",
        "assistant_type": "research"
    }
)

# Create LLM
llm = OpenAIChat(
    model="gpt-4",
    api_key="your-openai-api-key",
    base_url=PORTKEY_GATEWAY_URL,
    default_headers=portkey_headers
)

# Create assistant with tools
assistant = Assistant(
    llm=llm,
    description="You are a helpful research assistant",
    tools=[DuckDuckGo(), WikipediaTools()],
    show_tool_calls=True,
    markdown=True
)

# Use the assistant
assistant.print_response("What are the latest developments in fusion energy?")

Using Different Providers

Switch between LLM providers easily:

OpenAI GPT-4

from phi.llm.openai import OpenAIChat
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders

portkey_headers = createHeaders(
    api_key="your-portkey-api-key",
    provider="openai"
)

llm = OpenAIChat(
    model="gpt-4",
    api_key="your-openai-api-key",
    base_url=PORTKEY_GATEWAY_URL,
    default_headers=portkey_headers
)

assistant = Assistant(llm=llm)

Advanced Routing

Fallback Configuration

Automatically fallback to backup providers:

from phi.assistant import Assistant
from phi.llm.openai import OpenAIChat
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders

config = {
    "strategy": {"mode": "fallback"},
    "targets": [
        {"virtual_key": "openai-virtual-key"},
        {"virtual_key": "anthropic-virtual-key"},
        {"virtual_key": "together-virtual-key"}
    ]
}

portkey_headers = createHeaders(
    api_key="your-portkey-api-key",
    config=config
)

llm = OpenAIChat(
    model="gpt-4",
    api_key="X",  # Virtual keys in config
    base_url=PORTKEY_GATEWAY_URL,
    default_headers=portkey_headers
)

assistant = Assistant(
    llm=llm,
    description="Reliable assistant with fallbacks"
)

Load Balancing

Distribute requests across multiple models:

config = {
    "strategy": {"mode": "loadbalance"},
    "targets": [
        {
            "virtual_key": "openai-key-1",
            "weight": 0.7
        },
        {
            "virtual_key": "openai-key-2",
            "weight": 0.3
        }
    ]
}

portkey_headers = createHeaders(
    api_key="your-portkey-api-key",
    config=config
)

Retry Configuration

config = {
    "retry": {
        "attempts": 5,
        "on_status_codes": [429, 500, 502, 503]
    }
}

portkey_headers = createHeaders(
    api_key="your-portkey-api-key",
    provider="openai",
    config=config
)

Assistant with Memory

Create assistants with persistent memory:

from phi.assistant import Assistant
from phi.llm.openai import OpenAIChat
from phi.storage.assistant.postgres import PgAssistantStorage
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders

# Configure Portkey
portkey_headers = createHeaders(
    api_key="your-portkey-api-key",
    provider="openai",
    metadata={"feature": "memory"}
)

llm = OpenAIChat(
    model="gpt-4",
    api_key="your-openai-api-key",
    base_url=PORTKEY_GATEWAY_URL,
    default_headers=portkey_headers
)

# Create assistant with PostgreSQL storage
assistant = Assistant(
    llm=llm,
    storage=PgAssistantStorage(
        table_name="assistant_sessions",
        db_url="postgresql://user:pass@localhost:5432/ai"
    ),
    description="Assistant with memory",
    add_chat_history_to_messages=True,
    num_history_messages=4
)

# First conversation
assistant.print_response("My name is Alice")

# Later conversation - assistant remembers
assistant.print_response("What's my name?")

Knowledge Base Integration

Build assistants with knowledge bases:

from phi.assistant import Assistant
from phi.llm.openai import OpenAIChat
from phi.knowledge.pdf import PDFUrlKnowledgeBase
from phi.vectordb.pgvector import PgVector2
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders

# Configure Portkey
portkey_headers = createHeaders(
    api_key="your-portkey-api-key",
    provider="openai"
)

llm = OpenAIChat(
    model="gpt-4",
    api_key="your-openai-api-key",
    base_url=PORTKEY_GATEWAY_URL,
    default_headers=portkey_headers
)

# Create knowledge base
knowledge_base = PDFUrlKnowledgeBase(
    urls=["https://example.com/document.pdf"],
    vector_db=PgVector2(
        collection="pdf_documents",
        db_url="postgresql://user:pass@localhost:5432/ai"
    )
)

# Load knowledge base
knowledge_base.load(recreate=False)

# Create assistant with knowledge
assistant = Assistant(
    llm=llm,
    knowledge_base=knowledge_base,
    description="Assistant with document knowledge",
    add_references_to_prompt=True
)

assistant.print_response("What does the document say about AI?")

Using Phidata Tools

Integrate various tools with your assistant:

from phi.assistant import Assistant
from phi.llm.openai import OpenAIChat
from phi.tools.duckduckgo import DuckDuckGo
from phi.tools.yfinance import YFinanceTools
from phi.tools.arxiv_toolkit import ArxivToolkit
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders

portkey_headers = createHeaders(
    api_key="your-portkey-api-key",
    provider="openai"
)

llm = OpenAIChat(
    model="gpt-4",
    api_key="your-openai-api-key",
    base_url=PORTKEY_GATEWAY_URL,
    default_headers=portkey_headers
)

# Multi-tool assistant
assistant = Assistant(
    llm=llm,
    tools=[
        DuckDuckGo(),
        YFinanceTools(stock_price=True, analyst_recommendations=True),
        ArxivToolkit()
    ],
    description="Multi-purpose research assistant",
    show_tool_calls=True,
    markdown=True
)

assistant.print_response(
    "What is the current stock price of AAPL and find recent papers about AI?"
)

Caching for Assistants

Enable caching to reduce costs:

config = {
    "cache": {
        "mode": "semantic",
        "max_age": 3600  # 1 hour
    }
}

portkey_headers = createHeaders(
    api_key="your-portkey-api-key",
    provider="openai",
    config=config
)

llm = OpenAIChat(
    model="gpt-4",
    api_key="your-openai-api-key",
    base_url=PORTKEY_GATEWAY_URL,
    default_headers=portkey_headers
)

Streaming Responses

Stream assistant responses:

from phi.assistant import Assistant
from phi.llm.openai import OpenAIChat
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders

portkey_headers = createHeaders(
    api_key="your-portkey-api-key",
    provider="openai"
)

llm = OpenAIChat(
    model="gpt-4",
    api_key="your-openai-api-key",
    base_url=PORTKEY_GATEWAY_URL,
    default_headers=portkey_headers
)

assistant = Assistant(
    llm=llm,
    description="Streaming assistant"
)

# Stream response
for chunk in assistant.run("Tell me a long story", stream=True):
    print(chunk.content, end="", flush=True)

Team of Assistants

Create multiple specialized assistants:

from phi.assistant import Assistant
from phi.llm.openai import OpenAIChat
from phi.tools.duckduckgo import DuckDuckGo
from phi.tools.python import PythonTools
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders

portkey_headers = createHeaders(
    api_key="your-portkey-api-key",
    provider="openai"
)

llm = OpenAIChat(
    model="gpt-4",
    api_key="your-openai-api-key",
    base_url=PORTKEY_GATEWAY_URL,
    default_headers=portkey_headers
)

# Research assistant
researcher = Assistant(
    name="Researcher",
    llm=llm,
    description="Research information on the web",
    tools=[DuckDuckGo()],
    show_tool_calls=True
)

# Python assistant
python_assistant = Assistant(
    name="PythonDeveloper",
    llm=llm,
    description="Write and run Python code",
    tools=[PythonTools()],
    show_tool_calls=True
)

# Coordinator assistant
coordinator = Assistant(
    name="Coordinator",
    llm=llm,
    description="Coordinate between researchers and developers",
    team=[researcher, python_assistant],
    show_tool_calls=True
)

# Use the team
coordinator.print_response(
    "Research Python best practices and create a code example"
)

Observability and Tracking

Add detailed tracking to your assistants:

from phi.assistant import Assistant
from phi.llm.openai import OpenAIChat
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders

portkey_headers = createHeaders(
    api_key="your-portkey-api-key",
    provider="openai",
    metadata={
        "user_id": "user_123",
        "session_id": "session_456",
        "assistant_type": "customer_support",
        "environment": "production"
    },
    trace_id="assistant-conversation-001"
)

llm = OpenAIChat(
    model="gpt-4",
    api_key="your-openai-api-key",
    base_url=PORTKEY_GATEWAY_URL,
    default_headers=portkey_headers
)

assistant = Assistant(llm=llm)

View metrics in Portkey dashboard:

• Conversation flows
• Token usage per session
• Response latency
• Tool usage patterns
• Cache hit rates
• Error tracking

Best Practices

Use Fallbacks for Production

Configure fallback providers for reliability:

config = {"strategy": {"mode": "fallback"}, "targets": [...]}

Enable Caching

Use semantic caching for FAQ-style assistants:

config = {"cache": {"mode": "semantic", "max_age": 3600}}

Add Metadata

Track user sessions with metadata:

metadata={"user_id": "user_123", "session_id": "session_456"}

Monitor Token Usage

Use the Portkey dashboard to track and optimize token consumption.

Use Knowledge Bases

Integrate knowledge bases for domain-specific assistants to reduce hallucinations.

Example: Customer Support Assistant

Complete customer support assistant:

from phi.assistant import Assistant
from phi.llm.openai import OpenAIChat
from phi.tools.duckduckgo import DuckDuckGo
from phi.knowledge.pdf import PDFKnowledgeBase
from phi.vectordb.pgvector import PgVector2
from phi.storage.assistant.postgres import PgAssistantStorage
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders

# Configure Portkey with fallbacks
config = {
    "strategy": {"mode": "fallback"},
    "targets": [
        {"virtual_key": "openai-key"},
        {"virtual_key": "anthropic-key"}
    ],
    "cache": {
        "mode": "semantic",
        "max_age": 1800
    }
}

portkey_headers = createHeaders(
    api_key="your-portkey-api-key",
    config=config,
    metadata={"assistant": "customer_support"}
)

llm = OpenAIChat(
    model="gpt-4",
    api_key="X",
    base_url=PORTKEY_GATEWAY_URL,
    default_headers=portkey_headers
)

# Create knowledge base from documentation
knowledge_base = PDFKnowledgeBase(
    path="docs/",
    vector_db=PgVector2(
        collection="product_docs",
        db_url="postgresql://user:pass@localhost:5432/ai"
    )
)
knowledge_base.load()

# Create assistant with memory and knowledge
support_assistant = Assistant(
    name="SupportBot",
    llm=llm,
    knowledge_base=knowledge_base,
    storage=PgAssistantStorage(
        table_name="support_sessions",
        db_url="postgresql://user:pass@localhost:5432/ai"
    ),
    description="""You are a helpful customer support assistant.
    Use the knowledge base to answer questions accurately.
    Be friendly and professional.""",
    tools=[DuckDuckGo()],
    add_chat_history_to_messages=True,
    add_references_to_prompt=True,
    markdown=True,
    debug_mode=True
)

# Handle customer query
support_assistant.print_response(
    "How do I reset my password?",
    stream=True
)

Error Handling

Implement robust error handling:

from phi.assistant import Assistant
from phi.llm.openai import OpenAIChat
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders

config = {
    "retry": {"attempts": 3},
    "strategy": {"mode": "fallback"},
    "targets": [
        {"virtual_key": "openai-key"},
        {"virtual_key": "anthropic-key"}
    ]
}

portkey_headers = createHeaders(
    api_key="your-portkey-api-key",
    config=config
)

try:
    llm = OpenAIChat(
        model="gpt-4",
        api_key="X",
        base_url=PORTKEY_GATEWAY_URL,
        default_headers=portkey_headers
    )
    
    assistant = Assistant(llm=llm)
    response = assistant.run("Your query")
    print(response)
except Exception as e:
    print(f"Error: {e}")

Resources

• Phidata Documentation
• Portkey Gateway Configs
• Phidata Tools
• Example Notebooks

Need help? Join our Discord community for support with Phidata implementations.