Skip to content

LCORE-492: added tests for v1/conversations endpoints #646

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 9, 2025
Merged

LCORE-492: added tests for v1/conversations endpoints #646

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
227 changes: 174 additions & 53 deletions tests/e2e/features/conversations.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,54 +1,175 @@
# Feature: conversations endpoint API tests
#TODO: fix test

# Background:
# Given The service is started locally
# And REST API service hostname is localhost
# And REST API service port is 8080
# And REST API service prefix is /v1


# Scenario: Check if conversations endpoint finds the correct conversation when it exists
# Given The system is in default state
# When I access REST API endpoint "conversations" using HTTP GET method
# Then The status code of the response is 200
# And the proper conversation is returned

# Scenario: Check if conversations endpoint does not finds the conversation when it does not exists
# Given The system is in default state
# When I access REST API endpoint "conversations" using HTTP GET method
# Then The status code of the response is 404

# Scenario: Check if conversations endpoint fails when conversation id is not provided
# Given The system is in default state
# When I access REST API endpoint "conversations" using HTTP GET method
# Then The status code of the response is 422

# Scenario: Check if conversations endpoint fails when service is unavailable
# Given The system is in default state
# And the service is stopped
# When I access REST API endpoint "conversations" using HTTP GET method
# Then The status code of the response is 503

# Scenario: Check if conversations/delete endpoint finds the correct conversation when it exists
# Given The system is in default state
# When I access REST API endpoint "conversations/delete" using HTTP GET method
# Then The status code of the response is 200
# And the deleted conversation is not found

# Scenario: Check if conversations/delete endpoint does not finds the conversation when it does not exists
# Given The system is in default state
# When I access REST API endpoint "conversations/delete" using HTTP GET method
# Then The status code of the response is 404

# Scenario: Check if conversations/delete endpoint fails when conversation id is not provided
# Given The system is in default state
# When I access REST API endpoint "conversations/delete" using HTTP GET method
# Then The status code of the response is 422

# Scenario: Check if conversations/delete endpoint fails when service is unavailable
# Given The system is in default state
# And the service is stopped
# When I access REST API endpoint "conversations/delete" using HTTP GET method
# Then The status code of the response is 503
@Authorized
Feature: conversations endpoint API tests

Background:
Given The service is started locally
And REST API service hostname is localhost
And REST API service port is 8080
And REST API service prefix is /v1


Scenario: Check if conversations endpoint finds the correct conversation when it exists
Given The system is in default state
And I set the Authorization header to Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6Ikpva
And I use "query" to ask question with authorization header
"""
{"query": "Say hello", "model": "gpt-4-turbo", "provider": "openai"}
"""
And The status code of the response is 200
And I store conversation details
When I access REST API endpoint "conversations" using HTTP GET method
Then The status code of the response is 200
And The conversation with conversation_id from above is returned
And The conversation details are following
"""
{"last_used_model": "gpt-4-turbo", "last_used_provider": "openai", "message_count": 1}
"""

Scenario: Check if conversations endpoint fails when the auth header is not present
Given The system is in default state
And I set the Authorization header to Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6Ikpva
And I use "query" to ask question with authorization header
"""
{"query": "Say hello", "model": "gpt-4-turbo", "provider": "openai"}
"""
And The status code of the response is 200
And I store conversation details
And I remove the auth header
When I access REST API endpoint "conversations" using HTTP GET method
Then The status code of the response is 400
And The body of the response is the following
"""
{
"detail": "No Authorization header found"
}
"""

Scenario: Check if conversations/{conversation_id} endpoint finds the correct conversation when it exists
Given The system is in default state
And I set the Authorization header to Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6Ikpva
And I use "query" to ask question with authorization header
"""
{"query": "Say hello", "model": "gpt-4-turbo", "provider": "openai"}
"""
And The status code of the response is 200
And I store conversation details
When I use REST API conversation endpoint with conversation_id from above using HTTP GET method
Then The status code of the response is 200
And The returned conversation details have expected conversation_id
And The body of the response has following messages
"""
{"content": "Say hello", "type": "user", "content_response": "Hello", "type_response": "assistant"}
"""
And The body of the response has the following schema
"""
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"conversation_id": { "type": "string" },
"chat_history": {
"type": "array",
"items": {
"type": "object",
"properties": {
"messages": {
"type": "array",
"items": {
"type": "object",
"properties": {
"content": { "type": "string" },
"type": { "type": "string", "enum": ["user", "assistant"] }
}
}
},
"started_at": { "type": "string", "format": "date-time" },
"completed_at": { "type": "string", "format": "date-time" }
}
}
}
}
}
"""

Scenario: Check if conversations/{conversation_id} endpoint fails when the auth header is not present
Given The system is in default state
And I set the Authorization header to Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6Ikpva
And I use "query" to ask question with authorization header
"""
{"query": "Say hello", "model": "gpt-4-turbo", "provider": "openai"}
"""
And The status code of the response is 200
And I store conversation details
And I remove the auth header
When I use REST API conversation endpoint with conversation_id from above using HTTP GET method
Then The status code of the response is 400
And The body of the response is the following
"""
{
"detail": "No Authorization header found"
}
"""

Scenario: Check if conversations/{conversation_id} GET endpoint fails when conversation_id is malformed
Given The system is in default state
And I set the Authorization header to Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6Ikpva
When I use REST API conversation endpoint with conversation_id "abcdef" using HTTP GET method
Then The status code of the response is 422

Scenario: Check if conversations/{conversation_id} GET endpoint fails when llama-stack is unavailable
Given The system is in default state
And I set the Authorization header to Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6Ikpva
And I use "query" to ask question with authorization header
"""
{"query": "Say hello", "model": "gpt-4-turbo", "provider": "openai"}
"""
And The status code of the response is 200
And I store conversation details
And The llama-stack connection is disrupted
When I use REST API conversation endpoint with conversation_id from above using HTTP GET method
Then The status code of the response is 503
And The body of the response contains Unable to connect to Llama Stack

Scenario: Check if conversations DELETE endpoint removes the correct conversation
Given The system is in default state
And I set the Authorization header to Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6Ikpva
And I use "query" to ask question with authorization header
"""
{"query": "Say hello", "model": "gpt-4-turbo", "provider": "openai"}
"""
And The status code of the response is 200
And I store conversation details
When I use REST API conversation endpoint with conversation_id from above using HTTP DELETE method
Then The status code of the response is 200
And The returned conversation details have expected conversation_id
And The body of the response, ignoring the "conversation_id" field, is the following
"""
{"success": true, "response": "Conversation deleted successfully"}
"""
And I use REST API conversation endpoint with conversation_id from above using HTTP GET method
And The status code of the response is 404

Scenario: Check if conversations/{conversation_id} DELETE endpoint fails when conversation_id is malformed
Given The system is in default state
And I set the Authorization header to Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6Ikpva
When I use REST API conversation endpoint with conversation_id "abcdef" using HTTP DELETE method
Then The status code of the response is 422

Scenario: Check if conversations DELETE endpoint fails when the conversation does not exist
Given The system is in default state
When I use REST API conversation endpoint with conversation_id "12345678-abcd-0000-0123-456789abcdef" using HTTP DELETE method
Then The status code of the response is 404

Scenario: Check if conversations/{conversation_id} DELETE endpoint fails when llama-stack is unavailable
Given The system is in default state
And I set the Authorization header to Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6Ikpva
And I use "query" to ask question with authorization header
"""
{"query": "Say hello", "model": "gpt-4-turbo", "provider": "openai"}
"""
And The status code of the response is 200
And I store conversation details
And The llama-stack connection is disrupted
When I use REST API conversation endpoint with conversation_id from above using HTTP GET method
Then The status code of the response is 503
And The body of the response contains Unable to connect to Llama Stack
10 changes: 5 additions & 5 deletions tests/e2e/features/feedback.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Feature: feedback endpoint API tests
Given The system is in default state
When The feedback is enabled
Then The status code of the response is 200
And And the body of the response has the following structure
And the body of the response has the following structure
"""
{
"status":
Expand All @@ -27,7 +27,7 @@ Feature: feedback endpoint API tests
Given The system is in default state
When The feedback is disabled
Then The status code of the response is 200
And And the body of the response has the following structure
And the body of the response has the following structure
"""
{
"status":
Expand All @@ -46,7 +46,7 @@ Feature: feedback endpoint API tests
}
"""
Then The status code of the response is 422
And And the body of the response has the following structure
And the body of the response has the following structure
"""
{
"detail": [
Expand Down Expand Up @@ -123,7 +123,7 @@ Feature: feedback endpoint API tests
}
"""
Then The status code of the response is 422
And And the body of the response has the following structure
And the body of the response has the following structure
"""
{
"detail": [
Expand Down Expand Up @@ -211,7 +211,7 @@ Feature: feedback endpoint API tests
}
"""
Then The status code of the response is 422
And And the body of the response has the following structure
And the body of the response has the following structure
"""
{
"detail": [{
Expand Down
4 changes: 2 additions & 2 deletions tests/e2e/features/steps/common_http.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ def request_endpoint(context: Context, endpoint: str, hostname: str, port: int)
)


@then("The status code of the response is {status:d}")
@step("The status code of the response is {status:d}")
def check_status_code(context: Context, status: int) -> None:
"""Check the HTTP status code for latest response from tested service."""
assert context.response is not None, "Request needs to be performed first"
Expand Down Expand Up @@ -363,7 +363,7 @@ def check_for_null_attribute(context: Context, attribute: str) -> None:
), f"Attribute {attribute} should be null, but it contains {value}"


@then("And the body of the response has the following structure")
@then("the body of the response has the following structure")
def check_response_partially(context: Context) -> None:
"""Validate that the response body matches the expected JSON structure.

Expand Down
Loading
Loading