sonar-pro vs Web UI Discrepancy

[georgedickinson1]

The Perplexity API, when using the sonar-pro model, provides significantly less detailed and sometimes inaccurate information when tasked with extracting content from a specific target URL, compared to the Perplexity Web UI (Pro Search) operating on the same URL and query. This discrepancy persists even with configurations intended to maximize detail and focus, such as search_context_size: "high", a relevant search_domain_filter, and a highly specific system prompt emphasizing the single source URL.

The core issue is a failure of the API to match the Web UI's ability to extract nuanced and complete information from the specified webpage.

When querying the API with the sonar-pro model, configured with search_context_size: "high", a search_domain_filter for santanderforintermediaries.co.uk, and a system prompt directing it to solely and exclusively use the content from https://www.santanderforintermediaries.co.uk/products-and-criteria/mortgage-lending-criteria, the API response should exhibit a similar level of detail, accuracy, and completeness as the Perplexity Web UI Pro Search when it is focused on that exact URL for the same query.

Specifically, for the example query "do they accept foreign income?", the API should return:

• The correct, limited list of accepted foreign currencies (USD, EUR, CHF).
• The crucial condition that the sterling (GBP) equivalent of the foreign income must be calculated and then discounted by 25% to allow for currency fluctuations.
• The requirement to record the foreign currency amount, GBP equivalent, exchange rate on the day of calculation, and the date of calculation in the application notes.
• Other relevant conditions and notes as present on the target webpage and identified by the Web UI.

The API response is consistently less detailed and misses critical information for the example query "do they accept foreign income?" when targeting https://www.santanderforintermediaries.co.uk/products-and-criteria/mortgage-lending-criteria.

Specifically, the API's actual behaviour includes:

• Often providing a much broader and potentially inaccurate list of acceptable foreign currencies (e.g., including AUD, CAD, DKK, etc., beyond the correct USD, EUR, CHF).
• Critically and consistently omitting the 25% discount rule applied to the GBP equivalent of foreign income.
• Failing to mention the requirement to record specific calculation details in the application notes.
  -Generally providing a more summarised and less actionable response compared to the Web UI's Pro Search output.

This indicates a significant gap in the API's ability to deeply parse and accurately represent the content of the specified URL compared to the Web UI.

Steps to Reproduce:

1. Call the Perplexity API (https://api.perplexity.ai/chat/completions) with a request structured similarly to the payload described below (derived from the attached Python script example).

• Model: sonar-pro
• System Prompt: A detailed prompt (see "Additional Context" or attached script) heavily emphasising the TARGET_SANTANDER_URL (https://www.santanderforintermediaries.co.uk/products-and-criteria/mortgage-lending-criteria) as the sole source.
• User Query: "do they accept foreign income?"
• Parameters:
  • search_domain_filter: ["santanderforintermediaries.co.uk"]
  • web_search_options: { "search_context_size": "high", "search_recency_filter": "week" } (or other recency settings like "day", "last_month").
  • temperature: Tested with API default (omitted) and explicit values (e.g., 0.0, 0.2).
  • max_tokens: e.g., 3000.

2. Observe the API's JSON response, specifically the content of the assistant's message.

3. Compare this API output to the response obtained by:

• Navigating to the Perplexity Web UI.
• Ensuring Pro Search is active.
• Focusing the search on the URL: https://www.santanderforintermediaries.co.uk/products-and-criteria/mortgage-lending-criteria in the prompt.
• Posing the same query: "do they accept foreign income?".

4. Note the discrepancies in detail and accuracy, particularly the missing 25% discount rule and incorrect currency lists in the API response.

API Request & Response:
Example API Request Payload (Conceptual Structure):

{
"model": "sonar-pro",
"messages": [
{
"role": "system",
"content": "/* A highly detailed system prompt emphasising TARGET_SANTANDER_URL as the sole source. Key instruction: 'All information provided in your final answer MUST originate only from the content of the webpage: https://www.santanderforintermediaries.co.uk/products-and-criteria/mortgage-lending-criteria. Within the search results obtained from the santanderforintermediaries.co.uk domain... you MUST prioritize and extract information exclusively from content that directly corresponds to the https://www.santanderforintermediaries.co.uk/products-and-criteria/mortgage-lending-criteria page...' */"
},
{
"role": "user",
"content": "do they accept foreign income?"
}
],
"search_domain_filter": ["santanderforintermediaries.co.uk"],
"max_tokens": 3000,
// "temperature": 0.2, // (also tested with default/omitted)
"web_search_options": {
"search_context_size": "high",
"search_recency_filter": "week" // (also tested with "day", "last_month")
}
}

Example API Response (Actual - Summarised for "foreign income" query):

• Lists multiple currencies including AUD, CAD, DKK, EUR, HKD, JPY, NZD, NOK, SGD, SEK, CHF, USD.
• Mentions income verification and conversion to sterling.
• Mentions requirement for a UK bank account.
• MISSES the 25% discount rule.
• MISSES the specific requirement to record calculation details.

Example Web UI Pro Search Response (Expected - Summarised for "foreign income" query):

• Correctly states acceptance of employed income in USD, EUR, CHF.
• Clearly states the sterling (GBP) equivalent must be calculated and then discounted by 25%.
• States this discounted GBP figure must be entered in the application.
• States you must record foreign currency amount, GBP equivalent, exchange rate, and date of calculation in application notes.
• Includes other important notes like "Only employed income in the listed currencies is accepted..."

Environment:

• API Model Used: sonar-pro
• SDK (if applicable): Python requests library (direct HTTP calls, not a specific Perplexity SDK)
• Operating System: MacOS
• Authentication Type: API Key

Additional Context:
The primary goal is to use the Perplexity API for highly focused, accurate information retrieval from a single, specified URL, mirroring the capabilities observed in the Web UI Pro Search. The target URL for this report is https://www.santanderforintermediaries.co.uk/products-and-criteria/mortgage-lending-criteria.

The system prompt used is exceptionally detailed and explicitly commands the model to source information solely and exclusively from this specific URL, and to prioritize it even within the domain-filtered search results. Despite this, and experimentation with temperature, search_recency_filter, and ensuring search_context_size: "high", the API fails to extract critical details that the Web UI successfully retrieves.

This issue significantly impacts the reliability of the API for tasks requiring high-fidelity extraction from specific web documents. The problem seems to lie in the API's content fetching/processing pipeline for the given URL, or how the sonar-pro model interprets this fetched content, when compared to the Web UI's mechanisms.