api-inference.huggingface.co/models/xyz started giving 404

[sskanishk]

Calling inference api to call model for text generation.
But now getting 404 not found, this setup was working earlier. What's missing here? any new change rolled out that I'm not aware of?

Code:

HF_API_URL = f"https://api-inference.huggingface.co/models/mistralai/Mistral-7B-Instruct-v0.3"
HEADERS = {
    "Authorization": f"Bearer {settings.HF_TOKEN}"
}

async def query_hf_api(payload: dict) -> str:
    async with aiohttp.ClientSession() as session:
        async with session.post(
            HF_API_URL, 
            headers=HEADERS, 
            json=payload,
            timeout=30
        ) as response:
            if response.status != 200:
                error = await response.text()
                raise Exception(f"HF API error {response.status}: {error}")
                
            result = await response.json()
            return result[0]['generated_text']


    payload = {
        "inputs": prompt,
        "parameters": {
            "temperature": 0.3,
            "max_new_tokens": 1024,
            "return_full_text": False
        }
    }

response = await query_hf_api(payload)

Error:

<ClientResponse(https://api-inference.huggingface.co/models/meta-llama/Mistral-7B-Instruct-v0.3) [404 Not Found]>
<CIMultiDictProxy(
    'Content-Type': 'text/plain; charset=utf-8',
    'Content-Length': '9',
    'Connection': 'keep-alive',
    'Date': 'Mon, 07 Jul 2025 11:29:49 GMT',
    'Access-Control-Allow-Credentials': 'true',
    'X-Powered-By': 'huggingface-moon',
    'cross-origin-opener-policy': 'same-origin',
    'Referrer-Policy': 'strict-origin-when-cross-origin',
    'x-request-id': 'Root=1-686bafad-49b4c71a2c85be1316d2847d',
    'Vary': 'origin, access-control-request-method, access-control-request-headers',
    'Access-Control-Allow-Origin': '*',
    'Access-Control-Expose-Headers': 'X-Repo-Commit,X-Request-Id,X-Error-Code,X-Error-Message,X-Total-Count,ETag,Link,Accept-Ranges,Content-Range,X-Linked-Size,X-Linked-ETag,X-Xet-Hash',
    'X-Cache': 'Error from cloudfront',
    'Via': '1.1 9d889af515a5cd86a11x05f7befed532.cloudfront.net (CloudFront)',
    'X-Amz-Cf-Pop': 'DEL51-P5',
    'X-Amz-Cf-Id': 'x7JSW98lzRvThw5yEvCt4-LzVxo6vxv4jxpWkvBHrHQQm8ZoOlQVhQ=='
)>

[julien-c]

api-inference.huggingface.co is a very old URL, where did you find it documented?

[sskanishk]

@julien-c probably got it from some blog posts months ago. The endpoint still works without errors, but I’m happy to switch with recommended one.

Strange part - the issue might be not with this endpoint, because the same endpoint works fine when I use a different model.

Here's the code snippet below.

test.py

import asyncio
import aiohttp

### -----> works ok with sshleifer
# HF_API_URL = f"https://api-inference.huggingface.co/models/sshleifer/distilbart-cnn-12-6"

### -----> *not found* error response
HF_API_URL = f"https://api-inference.huggingface.co/models/mistralai/Mistral-7B-Instruct-v0.3"


API_TOKEN = '-----hf_token-----'

HEADERS = {
    "Authorization": f"Bearer {API_TOKEN}",
    "Content-Type": "application/json"
}

prompt = """
### Instruction:
Answer the question using ONLY the context below. Follow these rules:
1. Be factual and concise
2. If answer isn't in context, say "I don't have that information"
3. Never invent details

### Context:
DOCUMENT 1 (Similarity: 0.65):
Generative AI, often referred to as GenAI, is a subset of artificial intelligence that can create original 
content such as text, images, videos, audio, or software code in response to user prompts. This 
technology relies on sophisticated machine learning models, particularly deep learning models, 
which simulate the learning and decision-making processes of the human brain.


DOCUMENT 2 (Similarity: 0.65):
Generative AI, often referred to as GenAI, is a subset of artificial intelligence that can create original 
content such as text, images, videos, audio, or software code in response to user prompts. This 
technology relies on sophisticated machine learning models, particularly deep learning models, 
which simulate the learning and decision-making processes of the human brain.


DOCUMENT 3 (Similarity: 0.65):
Generative AI, often referred to as GenAI, is a subset of artificial intelligence that can create original 
content such as text, images, videos, audio, or software code in response to user prompts. This 
technology relies on sophisticated machine learning models, particularly deep learning models, 
which simulate the learning and decision-making processes of the human brain.


### Question:
what do you mean by Gen AI?

### Response:
"""

async def query_hf_api(payload: dict) -> str:
    async with aiohttp.ClientSession() as session:
        async with session.post(
            HF_API_URL, 
            headers=HEADERS, 
            json=payload,
            timeout=30
        ) as response:
            if response.status != 200:
                error = await response.text()
                raise Exception(f"HF API error {response.status}: {error}")
                
            result = await response.json()
            return result[0]

# Call using asyncio
if __name__ == "__main__":
    payload = {
        "inputs": prompt,
        "parameters": {
            "temperature": 0.3,
            "max_new_tokens": 50
        }
    }
    response = asyncio.run(query_hf_api(payload))
    print("Generated Text >>>>", response)