Skip to content

Commit d544e7f

Browse files
tisniktisnik
authored
Merge pull request #646 from radofuchs/conversation_v1_e2e_tests
LCORE-492: added tests for v1/conversations endpoints
2 parents 9c1ab9b + 558f22e commit d544e7f

File tree

4 files changed

+325
-68
lines changed

4 files changed

+325
-68
lines changed

‎tests/e2e/features/conversations.feature‎

Lines changed: 174 additions & 53 deletions
Original file line numberDiff line numberDiff line change
@@ -1,54 +1,175 @@
1-
# Feature: conversations endpoint API tests
2-
#TODO: fix test
3-
4-
# Background:
5-
# Given The service is started locally
6-
# And REST API service hostname is localhost
7-
# And REST API service port is 8080
8-
# And REST API service prefix is /v1
9-
10-
11-
# Scenario: Check if conversations endpoint finds the correct conversation when it exists
12-
# Given The system is in default state
13-
# When I access REST API endpoint "conversations" using HTTP GET method
14-
# Then The status code of the response is 200
15-
# And the proper conversation is returned
16-
17-
# Scenario: Check if conversations endpoint does not finds the conversation when it does not exists
18-
# Given The system is in default state
19-
# When I access REST API endpoint "conversations" using HTTP GET method
20-
# Then The status code of the response is 404
21-
22-
# Scenario: Check if conversations endpoint fails when conversation id is not provided
23-
# Given The system is in default state
24-
# When I access REST API endpoint "conversations" using HTTP GET method
25-
# Then The status code of the response is 422
26-
27-
# Scenario: Check if conversations endpoint fails when service is unavailable
28-
# Given The system is in default state
29-
# And the service is stopped
30-
# When I access REST API endpoint "conversations" using HTTP GET method
31-
# Then The status code of the response is 503
32-
33-
# Scenario: Check if conversations/delete endpoint finds the correct conversation when it exists
34-
# Given The system is in default state
35-
# When I access REST API endpoint "conversations/delete" using HTTP GET method
36-
# Then The status code of the response is 200
37-
# And the deleted conversation is not found
38-
39-
# Scenario: Check if conversations/delete endpoint does not finds the conversation when it does not exists
40-
# Given The system is in default state
41-
# When I access REST API endpoint "conversations/delete" using HTTP GET method
42-
# Then The status code of the response is 404
43-
44-
# Scenario: Check if conversations/delete endpoint fails when conversation id is not provided
45-
# Given The system is in default state
46-
# When I access REST API endpoint "conversations/delete" using HTTP GET method
47-
# Then The status code of the response is 422
48-
49-
# Scenario: Check if conversations/delete endpoint fails when service is unavailable
50-
# Given The system is in default state
51-
# And the service is stopped
52-
# When I access REST API endpoint "conversations/delete" using HTTP GET method
53-
# Then The status code of the response is 503
1+
@Authorized
2+
Feature: conversations endpoint API tests
543

4+
Background:
5+
Given The service is started locally
6+
And REST API service hostname is localhost
7+
And REST API service port is 8080
8+
And REST API service prefix is /v1
9+
10+
11+
Scenario: Check if conversations endpoint finds the correct conversation when it exists
12+
Given The system is in default state
13+
And I set the Authorization header to Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6Ikpva
14+
And I use "query" to ask question with authorization header
15+
"""
16+
{"query": "Say hello", "model": "gpt-4-turbo", "provider": "openai"}
17+
"""
18+
And The status code of the response is 200
19+
And I store conversation details
20+
When I access REST API endpoint "conversations" using HTTP GET method
21+
Then The status code of the response is 200
22+
And The conversation with conversation_id from above is returned
23+
And The conversation details are following
24+
"""
25+
{"last_used_model": "gpt-4-turbo", "last_used_provider": "openai", "message_count": 1}
26+
"""
27+
28+
Scenario: Check if conversations endpoint fails when the auth header is not present
29+
Given The system is in default state
30+
And I set the Authorization header to Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6Ikpva
31+
And I use "query" to ask question with authorization header
32+
"""
33+
{"query": "Say hello", "model": "gpt-4-turbo", "provider": "openai"}
34+
"""
35+
And The status code of the response is 200
36+
And I store conversation details
37+
And I remove the auth header
38+
When I access REST API endpoint "conversations" using HTTP GET method
39+
Then The status code of the response is 400
40+
And The body of the response is the following
41+
"""
42+
{
43+
"detail": "No Authorization header found"
44+
}
45+
"""
46+
47+
Scenario: Check if conversations/{conversation_id} endpoint finds the correct conversation when it exists
48+
Given The system is in default state
49+
And I set the Authorization header to Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6Ikpva
50+
And I use "query" to ask question with authorization header
51+
"""
52+
{"query": "Say hello", "model": "gpt-4-turbo", "provider": "openai"}
53+
"""
54+
And The status code of the response is 200
55+
And I store conversation details
56+
When I use REST API conversation endpoint with conversation_id from above using HTTP GET method
57+
Then The status code of the response is 200
58+
And The returned conversation details have expected conversation_id
59+
And The body of the response has following messages
60+
"""
61+
{"content": "Say hello", "type": "user", "content_response": "Hello", "type_response": "assistant"}
62+
"""
63+
And The body of the response has the following schema
64+
"""
65+
{
66+
"$schema": "https://json-schema.org/draft/2020-12/schema",
67+
"type": "object",
68+
"properties": {
69+
"conversation_id": { "type": "string" },
70+
"chat_history": {
71+
"type": "array",
72+
"items": {
73+
"type": "object",
74+
"properties": {
75+
"messages": {
76+
"type": "array",
77+
"items": {
78+
"type": "object",
79+
"properties": {
80+
"content": { "type": "string" },
81+
"type": { "type": "string", "enum": ["user", "assistant"] }
82+
}
83+
}
84+
},
85+
"started_at": { "type": "string", "format": "date-time" },
86+
"completed_at": { "type": "string", "format": "date-time" }
87+
}
88+
}
89+
}
90+
}
91+
}
92+
"""
93+
94+
Scenario: Check if conversations/{conversation_id} endpoint fails when the auth header is not present
95+
Given The system is in default state
96+
And I set the Authorization header to Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6Ikpva
97+
And I use "query" to ask question with authorization header
98+
"""
99+
{"query": "Say hello", "model": "gpt-4-turbo", "provider": "openai"}
100+
"""
101+
And The status code of the response is 200
102+
And I store conversation details
103+
And I remove the auth header
104+
When I use REST API conversation endpoint with conversation_id from above using HTTP GET method
105+
Then The status code of the response is 400
106+
And The body of the response is the following
107+
"""
108+
{
109+
"detail": "No Authorization header found"
110+
}
111+
"""
112+
113+
Scenario: Check if conversations/{conversation_id} GET endpoint fails when conversation_id is malformed
114+
Given The system is in default state
115+
And I set the Authorization header to Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6Ikpva
116+
When I use REST API conversation endpoint with conversation_id "abcdef" using HTTP GET method
117+
Then The status code of the response is 422
118+
119+
Scenario: Check if conversations/{conversation_id} GET endpoint fails when llama-stack is unavailable
120+
Given The system is in default state
121+
And I set the Authorization header to Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6Ikpva
122+
And I use "query" to ask question with authorization header
123+
"""
124+
{"query": "Say hello", "model": "gpt-4-turbo", "provider": "openai"}
125+
"""
126+
And The status code of the response is 200
127+
And I store conversation details
128+
And The llama-stack connection is disrupted
129+
When I use REST API conversation endpoint with conversation_id from above using HTTP GET method
130+
Then The status code of the response is 503
131+
And The body of the response contains Unable to connect to Llama Stack
132+
133+
Scenario: Check if conversations DELETE endpoint removes the correct conversation
134+
Given The system is in default state
135+
And I set the Authorization header to Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6Ikpva
136+
And I use "query" to ask question with authorization header
137+
"""
138+
{"query": "Say hello", "model": "gpt-4-turbo", "provider": "openai"}
139+
"""
140+
And The status code of the response is 200
141+
And I store conversation details
142+
When I use REST API conversation endpoint with conversation_id from above using HTTP DELETE method
143+
Then The status code of the response is 200
144+
And The returned conversation details have expected conversation_id
145+
And The body of the response, ignoring the "conversation_id" field, is the following
146+
"""
147+
{"success": true, "response": "Conversation deleted successfully"}
148+
"""
149+
And I use REST API conversation endpoint with conversation_id from above using HTTP GET method
150+
And The status code of the response is 404
151+
152+
Scenario: Check if conversations/{conversation_id} DELETE endpoint fails when conversation_id is malformed
153+
Given The system is in default state
154+
And I set the Authorization header to Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6Ikpva
155+
When I use REST API conversation endpoint with conversation_id "abcdef" using HTTP DELETE method
156+
Then The status code of the response is 422
157+
158+
Scenario: Check if conversations DELETE endpoint fails when the conversation does not exist
159+
Given The system is in default state
160+
When I use REST API conversation endpoint with conversation_id "12345678-abcd-0000-0123-456789abcdef" using HTTP DELETE method
161+
Then The status code of the response is 404
162+
163+
Scenario: Check if conversations/{conversation_id} DELETE endpoint fails when llama-stack is unavailable
164+
Given The system is in default state
165+
And I set the Authorization header to Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6Ikpva
166+
And I use "query" to ask question with authorization header
167+
"""
168+
{"query": "Say hello", "model": "gpt-4-turbo", "provider": "openai"}
169+
"""
170+
And The status code of the response is 200
171+
And I store conversation details
172+
And The llama-stack connection is disrupted
173+
When I use REST API conversation endpoint with conversation_id from above using HTTP GET method
174+
Then The status code of the response is 503
175+
And The body of the response contains Unable to connect to Llama Stack

‎tests/e2e/features/feedback.feature‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ Feature: feedback endpoint API tests
1313
Given The system is in default state
1414
When The feedback is enabled
1515
Then The status code of the response is 200
16-
And And the body of the response has the following structure
16+
And the body of the response has the following structure
1717
"""
1818
{
1919
"status":
@@ -27,7 +27,7 @@ Feature: feedback endpoint API tests
2727
Given The system is in default state
2828
When The feedback is disabled
2929
Then The status code of the response is 200
30-
And And the body of the response has the following structure
30+
And the body of the response has the following structure
3131
"""
3232
{
3333
"status":
@@ -46,7 +46,7 @@ Feature: feedback endpoint API tests
4646
}
4747
"""
4848
Then The status code of the response is 422
49-
And And the body of the response has the following structure
49+
And the body of the response has the following structure
5050
"""
5151
{
5252
"detail": [
@@ -123,7 +123,7 @@ Feature: feedback endpoint API tests
123123
}
124124
"""
125125
Then The status code of the response is 422
126-
And And the body of the response has the following structure
126+
And the body of the response has the following structure
127127
"""
128128
{
129129
"detail": [
@@ -211,7 +211,7 @@ Feature: feedback endpoint API tests
211211
}
212212
"""
213213
Then The status code of the response is 422
214-
And And the body of the response has the following structure
214+
And the body of the response has the following structure
215215
"""
216216
{
217217
"detail": [{

‎tests/e2e/features/steps/common_http.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -115,7 +115,7 @@ def request_endpoint(context: Context, endpoint: str, hostname: str, port: int)
115115
)
116116

117117

118-
@then("The status code of the response is {status:d}")
118+
@step("The status code of the response is {status:d}")
119119
def check_status_code(context: Context, status: int) -> None:
120120
"""Check the HTTP status code for latest response from tested service."""
121121
assert context.response is not None, "Request needs to be performed first"
@@ -363,7 +363,7 @@ def check_for_null_attribute(context: Context, attribute: str) -> None:
363363
), f"Attribute {attribute} should be null, but it contains {value}"
364364

365365

366-
@then("And the body of the response has the following structure")
366+
@then("the body of the response has the following structure")
367367
def check_response_partially(context: Context) -> None:
368368
"""Validate that the response body matches the expected JSON structure.
369369

0 commit comments

Comments
 (0)