Skip to content

Commit a5971d1

Browse files
Knowledge db (#196)
* wip: thread stream. * wip(chat-db): add middleware to connect to db and get threads. * fix(jb images): remove "add image" button from jetbrains. (#211) * fix(jb images): remove "add image" button from jetbrains. * fix(jb images): make `images` a configurable feature. * wip(knowledge): add sub end point. * wip(knowlage db): handle insert, update and delete operations. * wip(memories): add a basic ui for rendering memories. * wip(knowledge): add create, delete and search services. * wip(knowledge): add url for updating usage. * wip(knowledge): add the knowledge page to the sidebar. * wip(knowledge): add a basic form for adding knowledge. * wip(knowledge form): add back button. * wip(knowledge): add back in listAll and add a scroll area to memories list. * wip(knowledge): list all and subscribe * wip(knowledge): add delete functionality. * ui(knowledge): change the orientation of the data-list depending on size * wip(knowledge list): remove properties the user doesn't need to see from the data list. * ui(knowledge list): remove data list and use `m_goal` as the title and `payload` as content. * feat(knowledge): add a like button. * ui(thumb): change like button from tick to thumb. * refactor(knowledge): remove project, type and origin from add knowledge form * refactor(knowledge): remove `mem-list` endpoint. * feat(knowledge): hide behind a feature flag. * feat(feature menu): add a hidden feature menu. * wip(knowledge): add the ability to edit knowledge. * fix(feature menu): only allow the code to be entered when nothing if focused. * wip(knowledge search): add comments and rough search handler * feat(knowledge): add quick search and fix features usage * fix(features): add links to setting for features set in settings. * chore*knowledge): fix linter warnings * wip(knowledge): add `mem-upd` handler for updating memroies. * wip(knowledge): add status to subscription payload. * wip(knowledge): add vectorisation status information. * wip(knowledge): animate db button while parsing. * wip(knowledge): debouce search and cache vecdb status. * refactor(knowledge): move components to separate files. * ui(knowledge): add titles to the forms. * ui(knowledge): add padding to items so the scroll bar isn't over the boarder * linter: allow log for unsupproted type in knowledge stream. * fix(knowledge): don't wait for consumeStream to resolve, this allows the stream to hangup. * fix: refetch button if counter state is disaligned on frontend with inference * fix: user.refetch instead of reset of API state * fix: updating state of agent usage on refetch of user * fix: void expression for async refetchUser * 2.0.3-alpha.25 * 2.0.3 * feat(knowledge): create knowledge from chat. * wip(knowledge): add a reducer and event listener for the current project name to pass to "/v1/trajectory-save". * wip(knowledge): export active_project types for vscode. * fix(knowledge): if the chat is an integration use the project from the integration data. * wip(knowledge): add a story with a knowledge tool call. * wip(knowledge): add basic ui for updating knowledge useage. * fix(knowledge): hang up on unmount. * wip(knowledge): move subscribe to a thunk so it can hangup requests. * 2.0.4-alpha.0 * fix(caps): refetch caps on mount or arg change. (#307) * fix: unified limit exhausted message across app & adjusted text * fix: better wording * 2.0.4-alpha.1 * fix: updated offline message * fix: increased size for input field * fix: if only one chat tab left and active tab is dashboard, allowing click on chat tab * feat: upgrade to pro button in main dropdown & reworked links in dropdown * fix: show plugin settings if jetbrains, otherwise extension settings * chore: revert deletion of integrations * chore: revert deletion of integrations * fix: startup keyword shouldn't be null * wip(updated tour): integrations step & updated refs * feat: updated onboarding with new data & small adjustments to the UI * 2.0.4-alpha.2 * wip(checkpoints): ui for popup and user input * wip(checkpoints): truncation of links for files * wip(checkpoints): storing checkpoints for user messages * wip(checkpoints): API handler, typeguards, saving restore results to store and fixed fixtures * wip(checkpoints): fixed types issue * wip(checkpoints): fix of tests * wip(checkpoints): stubs for checkpoints & storybook * feat(checkpoints): undo functionality & refactoring * chore: fixed types for CI * refactoring(checkpoints): extracted logic and functions to useCheckpoints hook * chore: parsing error text adjusted * fix: make checkpoints optional & hiding revert changes button if there are no checkpoints * fix: support esc key press & click out of the dialog to close it * chore: removed convertUTCToLocalTime() util * fix: remove hardcoded locale en-GB to use system one * feat: reworked logic & improved UI/UX * chore: better wording with dates * fix: checking for index of of current message instead of checking for user messages amount * fix(onboarding): updated GIF and links for final screen * fix: updated tour ending * 2.0.4-alpha.3 * fix: fixed validation of paths for puzzle links on Windows, adjusted texts * improve(puzzle_links): extractLinkFromPuzzle unit tests coverage * fix: tour blinking on restart & typos & unified titles for buttons & removed conditions for set up integrations button * 2.0.4-alpha.4 * 2.0.4 * 2.0.5-alpha.0 * Feat: Agent Capabilities (#321) * feat: better controls for agentic features * wip(chat-db): add knowledge sub thunk. * fix: clear memery when make a new request * wip(knowledge): move search to a thunk. * Login rework (#320) * 2.0.4-alpha.0 * fix(caps): refetch caps on mount or arg change. (#307) * fix: unified limit exhausted message across app & adjusted text * fix: better wording * 2.0.4-alpha.1 * fix: updated offline message * fix: increased size for input field * fix: if only one chat tab left and active tab is dashboard, allowing click on chat tab * feat: upgrade to pro button in main dropdown & reworked links in dropdown * fix: show plugin settings if jetbrains, otherwise extension settings * chore: revert deletion of integrations * chore: revert deletion of integrations * fix: startup keyword shouldn't be null * wip(updated tour): integrations step & updated refs * feat: updated onboarding with new data & small adjustments to the UI * 2.0.4-alpha.2 * wip(login): add un-themed accordion. * wip(accordion): light mode style * wip(accordion): support light and dark mode. * ui(accordion): use theme colour. * wip(login): add basic ui for login page. * ui(login): add padding to the container for the login page. * wip(login): add msw handlers for story book. * wip(checkpoints): ui for popup and user input * wip(checkpoints): truncation of links for files * wip(checkpoints): storing checkpoints for user messages * wip(checkpoints): API handler, typeguards, saving restore results to store and fixed fixtures * wip(checkpoints): fixed types issue * wip(checkpoints): fix of tests * wip(checkpoints): stubs for checkpoints & storybook * feat(checkpoints): undo functionality & refactoring * chore: fixed types for CI * refactoring(checkpoints): extracted logic and functions to useCheckpoints hook * chore: parsing error text adjusted * fix: make checkpoints optional & hiding revert changes button if there are no checkpoints * fix: support esc key press & click out of the dialog to close it * chore: removed convertUTCToLocalTime() util * fix: remove hardcoded locale en-GB to use system one * feat: reworked logic & improved UI/UX * chore: better wording with dates * fix: checking for index of of current message instead of checking for user messages amount * fix(onboarding): updated GIF and links for final screen * fix: updated tour ending * 2.0.4-alpha.3 * wip(email login): add hooks and msw interception for storybook * ui(login): small ui adjustments. * wip(login): handle canceling email polling. * fix(email login): remember to set api key on success. * wip(login): send setup host event to ide. * fix(email login redirect): quick fix, will need to look closer at App.tsx routing. * fix: fixed validation of paths for puzzle links on Windows, adjusted texts * improve(puzzle_links): extractLinkFromPuzzle unit tests coverage * fix: tour blinking on restart & typos & unified titles for buttons & removed conditions for set up integrations button * wip(login): remove old login pages. * feat: better controls for agentic features * test(start enw chat): for some reason stubResizeObserver wasn't being called :/ * wip(login): add provider icons. * refactor(agent usage): move usage update to getUser hook so email login updates the usage. * ui(login): add title. * fix(email message): add missing hooks. * fix(console.log): remove console log. * ui(login): fix ui jumping when changing text. * ui(login): allow all items to be collapsed. * ui(login): remove text changes because of jumps. --------- Co-authored-by: alashchev17 <[email protected]> * refactor: remove old chat-db code. * fix: preventing user from re-navigation, if history, welcome or chat pages are already in pages history * Feature: Deepseek Reasoning support (#312) * 2.0.4-alpha.0 * fix(caps): refetch caps on mount or arg change. (#307) * fix: unified limit exhausted message across app & adjusted text * fix: better wording * 2.0.4-alpha.1 * fix: updated offline message * fix: increased size for input field * fix: if only one chat tab left and active tab is dashboard, allowing click on chat tab * feat: upgrade to pro button in main dropdown & reworked links in dropdown * fix: show plugin settings if jetbrains, otherwise extension settings * chore: revert deletion of integrations * chore: revert deletion of integrations * fix: startup keyword shouldn't be null * wip(updated tour): integrations step & updated refs * feat: updated onboarding with new data & small adjustments to the UI * 2.0.4-alpha.2 * feat: THINKING_AGENT mode support for deepseek-reasoning model * wip(checkpoints): ui for popup and user input * wip(checkpoints): truncation of links for files * wip(checkpoints): storing checkpoints for user messages * wip(checkpoints): API handler, typeguards, saving restore results to store and fixed fixtures * wip(checkpoints): fixed types issue * wip(checkpoints): fix of tests * wip(checkpoints): stubs for checkpoints & storybook * feat(checkpoints): undo functionality & refactoring * chore: fixed types for CI * refactoring(checkpoints): extracted logic and functions to useCheckpoints hook * chore: parsing error text adjusted * fix: make checkpoints optional & hiding revert changes button if there are no checkpoints * fix: support esc key press & click out of the dialog to close it * chore: removed convertUTCToLocalTime() util * fix: remove hardcoded locale en-GB to use system one * feat: reworked logic & improved UI/UX * chore: better wording with dates * fix: checking for index of of current message instead of checking for user messages amount * fix(onboarding): updated GIF and links for final screen * fix: updated tour ending * 2.0.4-alpha.3 * fix: fixed validation of paths for puzzle links on Windows, adjusted texts * improve(puzzle_links): extractLinkFromPuzzle unit tests coverage * fix: tour blinking on restart & typos & unified titles for buttons & removed conditions for set up integrations button * feat: better controls for agentic features * feat: showing deepseek reasoner option only for PRO users, placing it in a correct place of all agentic features * chore: fix component file from merge conflicts * feat: better wording & hiding agent features if restored thread tool use is not agent * chore: better semantic name for reasoning switch component * fix: useGetUser to get user instead of outdated useLogin * 2.0.5-alpha.1 * ui(knowledge): move like button to bottom of last assistant message. * fix: cut off for mock data in agent rollback * 2.0.5-alpha.2 * wip(knowledge): make tool button look nicer. * wip(knowledge): add spacing around tool content. * wip(knowledge): add msw handler for updating usage. --------- Co-authored-by: alashchev17 <[email protected]> Co-authored-by: Andrii Lashchov <[email protected]>
1 parent 314486f commit a5971d1

40 files changed

+2592
-28
lines changed

src/__fixtures__/chat.ts

Lines changed: 570 additions & 0 deletions
Large diffs are not rendered by default.

src/__fixtures__/knowledge.ts

Lines changed: 260 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,260 @@
1+
import { MemdbSubEvent, MemoRecord, VecDbStatus } from "../events";
2+
3+
export const STUB_MEMORIES: MemoRecord[] = [
4+
{
5+
memid: "7666487b81",
6+
m_type: "proj-fact",
7+
m_goal: "compile",
8+
m_project: "proj1",
9+
m_payload: "Looks like proj1 is written in fact in Rust.",
10+
m_origin: "local-committed",
11+
mstat_correct: 1,
12+
mstat_relevant: -1,
13+
mstat_times_used: 1,
14+
},
15+
{
16+
memid: "cdec854819",
17+
m_type: "seq-of-acts",
18+
m_goal: "compile",
19+
m_project: "proj2",
20+
m_payload: "Wow, running cargo build on proj2 was successful!",
21+
m_origin: "local-committed",
22+
mstat_correct: 0,
23+
mstat_relevant: 0,
24+
mstat_times_used: 0,
25+
},
26+
{
27+
memid: "eb1d64684b",
28+
m_type: "proj-fact",
29+
m_goal: "compile",
30+
m_project: "proj2",
31+
m_payload: "Looks like proj2 is written in fact in Rust.",
32+
m_origin: "local-committed",
33+
mstat_correct: 0,
34+
mstat_relevant: 0,
35+
mstat_times_used: 0,
36+
},
37+
38+
{
39+
memid: "eb1d64684c",
40+
m_type: "proj-fact",
41+
m_goal:
42+
"Long goal Long goal Long goal Long goal Long goal Long goal Long goal Long goal Long goal Long goal",
43+
m_project: "proj2",
44+
m_payload: "Looks like proj2 is written in fact in Rust.",
45+
m_origin: "local-committed",
46+
mstat_correct: 0,
47+
mstat_relevant: 0,
48+
mstat_times_used: 0,
49+
},
50+
];
51+
52+
export const STUB_SUB_RESPONSE: MemdbSubEvent[] = [
53+
{
54+
pubevent_id: 19,
55+
pubevent_action: "INSERT",
56+
pubevent_json: {
57+
memid: "66a072d699",
58+
m_type: "seq-of-acts",
59+
m_goal: "compile",
60+
m_project: "proj1",
61+
m_payload: "Wow, running cargo build on proj1 was successful!",
62+
m_origin: "local-committed",
63+
mstat_correct: 0.0,
64+
mstat_relevant: 0.0,
65+
mstat_times_used: 0,
66+
},
67+
},
68+
{
69+
pubevent_id: 26,
70+
pubevent_action: "INSERT",
71+
pubevent_json: {
72+
memid: "d688925823",
73+
m_type: "proj-fact",
74+
m_goal: "compile",
75+
m_project: "proj1",
76+
m_payload: "Looks like proj1 is written in fact in Rust.",
77+
m_origin: "local-committed",
78+
mstat_correct: 0.0,
79+
mstat_relevant: 0.0,
80+
mstat_times_used: 0,
81+
},
82+
},
83+
{
84+
pubevent_id: 27,
85+
pubevent_action: "INSERT",
86+
pubevent_json: {
87+
memid: "08f9374753",
88+
m_type: "seq-of-acts",
89+
m_goal: "compile",
90+
m_project: "proj2",
91+
m_payload: "Wow, running cargo build on proj2 was successful!",
92+
m_origin: "local-committed",
93+
mstat_correct: 0.0,
94+
mstat_relevant: 0.0,
95+
mstat_times_used: 0,
96+
},
97+
},
98+
{
99+
pubevent_id: 28,
100+
pubevent_action: "INSERT",
101+
pubevent_json: {
102+
memid: "c9cefe3ff4",
103+
m_type: "proj-fact",
104+
m_goal: "compile",
105+
m_project: "proj2",
106+
m_payload: "Looks like proj2 is written in fact in Rust.",
107+
m_origin: "local-committed",
108+
mstat_correct: 0.0,
109+
mstat_relevant: 0.0,
110+
mstat_times_used: 0,
111+
},
112+
},
113+
{
114+
pubevent_id: 29,
115+
pubevent_action: "UPDATE",
116+
pubevent_json: {
117+
memid: "d688925823",
118+
m_type: "proj-fact",
119+
m_goal: "compile",
120+
m_project: "proj1",
121+
m_payload: "Looks like proj1 is written in fact in Rust.",
122+
m_origin: "local-committed",
123+
mstat_correct: 1.0,
124+
mstat_relevant: -1.0,
125+
mstat_times_used: 1,
126+
},
127+
},
128+
{
129+
pubevent_id: 30,
130+
pubevent_action: "DELETE",
131+
pubevent_json: {
132+
memid: "9d2a679b09",
133+
m_type: "",
134+
m_goal: "",
135+
m_project: "",
136+
m_payload: "",
137+
m_origin: "",
138+
mstat_correct: 0,
139+
mstat_relevant: 0,
140+
mstat_times_used: 0,
141+
},
142+
},
143+
];
144+
145+
// export const STUB_SUB_RESPONSE_WITH_STATUS: (
146+
// | MemdbSubEventUnparsed
147+
// | VecDbStatus
148+
// )[] = [];
149+
150+
export const VECDB_STATUS_STARTING: VecDbStatus = {
151+
files_unprocessed: 0,
152+
files_total: 0,
153+
requests_made_since_start: 1,
154+
vectors_made_since_start: 33,
155+
db_size: 33,
156+
db_cache_size: 37,
157+
state: "starting",
158+
queue_additions: false,
159+
vecdb_max_files_hit: false,
160+
vecdb_errors: {},
161+
};
162+
163+
export const VECDB_STATUS_PARSING: VecDbStatus = {
164+
files_unprocessed: 377,
165+
files_total: 404,
166+
requests_made_since_start: 5,
167+
vectors_made_since_start: 296,
168+
db_size: 168,
169+
db_cache_size: 333,
170+
state: "parsing",
171+
queue_additions: false,
172+
vecdb_max_files_hit: false,
173+
vecdb_errors: {},
174+
};
175+
176+
export const VECDB_STATUS_PARSING_2: VecDbStatus = {
177+
files_unprocessed: 372,
178+
files_total: 404,
179+
requests_made_since_start: 6,
180+
vectors_made_since_start: 303,
181+
db_size: 303,
182+
db_cache_size: 340,
183+
state: "parsing",
184+
queue_additions: false,
185+
vecdb_max_files_hit: false,
186+
vecdb_errors: {},
187+
};
188+
189+
export const VECDV_STATUS_PARISING_3: VecDbStatus = {
190+
files_unprocessed: 192,
191+
files_total: 404,
192+
requests_made_since_start: 21,
193+
vectors_made_since_start: 990,
194+
db_size: 1021,
195+
db_cache_size: 1027,
196+
state: "parsing",
197+
queue_additions: false,
198+
vecdb_max_files_hit: false,
199+
vecdb_errors: {},
200+
};
201+
202+
export const VECDB_STATUS_PARSING_4: VecDbStatus = {
203+
files_unprocessed: 12,
204+
files_total: 404,
205+
requests_made_since_start: 52,
206+
vectors_made_since_start: 2494,
207+
db_size: 2524,
208+
db_cache_size: 2531,
209+
state: "parsing",
210+
queue_additions: false,
211+
vecdb_max_files_hit: false,
212+
vecdb_errors: {},
213+
};
214+
215+
export const VECDB_STATUS_COOLDOWN: VecDbStatus = {
216+
files_unprocessed: 1,
217+
files_total: 404,
218+
requests_made_since_start: 52,
219+
vectors_made_since_start: 2494,
220+
db_size: 2524,
221+
db_cache_size: 2533,
222+
state: "cooldown",
223+
queue_additions: false,
224+
vecdb_max_files_hit: false,
225+
vecdb_errors: {},
226+
};
227+
228+
export const VECDB_STATUS_DONE: VecDbStatus = {
229+
files_unprocessed: 0,
230+
files_total: 0,
231+
requests_made_since_start: 54,
232+
vectors_made_since_start: 2535,
233+
db_size: 2629,
234+
db_cache_size: 2574,
235+
state: "done",
236+
queue_additions: false,
237+
vecdb_max_files_hit: false,
238+
vecdb_errors: {},
239+
};
240+
241+
export const STUB_SUB_RESPONSE_WITH_STATUS = [
242+
VECDB_STATUS_STARTING,
243+
// ...STUB_SUB_RESPONSE,
244+
VECDB_STATUS_PARSING,
245+
VECDB_STATUS_PARSING_2,
246+
VECDV_STATUS_PARISING_3,
247+
VECDB_STATUS_PARSING_4,
248+
VECDB_STATUS_COOLDOWN,
249+
VECDB_STATUS_DONE,
250+
];
251+
252+
export const STB_LOADING_VECDB = {
253+
VECDB_STATUS_STARTING,
254+
VECDB_STATUS_PARSING,
255+
VECDB_STATUS_PARSING_2,
256+
VECDV_STATUS_PARISING_3,
257+
VECDB_STATUS_PARSING_4,
258+
VECDB_STATUS_COOLDOWN,
259+
VECDB_STATUS_DONE,
260+
};

src/__fixtures__/msw.ts

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,15 @@ import { STUB_LINKS_FOR_CHAT_RESPONSE } from "./chat_links_response";
55
import {
66
AT_TOOLS_AVAILABLE_URL,
77
CHAT_LINKS_URL,
8+
KNOWLEDGE_CREATE_URL,
9+
KNOWLEDGE_SUB_URL,
10+
KNOWLEDGE_UPDATE_USED_URL,
811
} from "../services/refact/consts";
912
import { STUB_TOOL_RESPONSE } from "./tools_response";
13+
import { STUB_SUB_RESPONSE, STUB_SUB_RESPONSE_WITH_STATUS } from "./knowledge";
1014
import { GoodPollingResponse } from "../services/smallcloud";
15+
import type { LinksForChatResponse } from "../services/refact/links";
16+
import { CompressTrajectoryResponse } from "../services/refact/knowledge";
1117

1218
export const goodPing: HttpHandler = http.get(
1319
"http://127.0.0.1:8001/v1/ping",
@@ -103,13 +109,106 @@ export const chatLinks: HttpHandler = http.post(
103109
},
104110
);
105111

112+
export const noChatLinks: HttpHandler = http.post(
113+
`http://127.0.0.1:8001${CHAT_LINKS_URL}`,
114+
() => {
115+
const res: LinksForChatResponse = {
116+
uncommited_changes_warning: "",
117+
links: [],
118+
};
119+
return HttpResponse.json(res);
120+
},
121+
);
122+
106123
export const goodTools: HttpHandler = http.get(
107124
`http://127.0.0.1:8001${AT_TOOLS_AVAILABLE_URL}`,
108125
() => {
109126
return HttpResponse.json(STUB_TOOL_RESPONSE);
110127
},
111128
);
112129

130+
export const knowLedgeLoading: HttpHandler = http.post(
131+
`http://127.0.0.1:8001${KNOWLEDGE_SUB_URL}`,
132+
async () => {
133+
const encoder = new TextEncoder();
134+
const stream = new ReadableStream({
135+
start(controller) {
136+
// Encode the string chunks using "TextEncoder".
137+
STUB_SUB_RESPONSE.forEach((item) => {
138+
const str = `data: ${JSON.stringify(item)}\n\n`;
139+
controller.enqueue(encoder.encode(str));
140+
});
141+
142+
controller.close();
143+
},
144+
});
145+
146+
await new Promise((resolve) => setTimeout(resolve, 3000));
147+
148+
return new HttpResponse(stream, {
149+
headers: {
150+
"Content-Type": "text/plain",
151+
},
152+
});
153+
},
154+
);
155+
156+
export const KnowledgeWithStatus: HttpHandler = http.post(
157+
`http://127.0.0.1:8001${KNOWLEDGE_SUB_URL}`,
158+
() => {
159+
const encoder = new TextEncoder();
160+
const stream = new ReadableStream({
161+
async start(controller) {
162+
// Encode the string chunks using "TextEncoder".
163+
for (const item of STUB_SUB_RESPONSE_WITH_STATUS) {
164+
const str = `data: ${JSON.stringify(item)}\n\n`;
165+
controller.enqueue(encoder.encode(str));
166+
await new Promise((resolve) => setTimeout(resolve, 3000)); // 1-second delay
167+
}
168+
169+
controller.close();
170+
},
171+
});
172+
173+
return new HttpResponse(stream, {
174+
headers: {
175+
"Content-Type": "text/plain",
176+
},
177+
});
178+
},
179+
);
180+
181+
export const makeKnowledgeFromChat: HttpHandler = http.post(
182+
`http://127.0.0.1:8001${KNOWLEDGE_CREATE_URL}`,
183+
() => {
184+
const result: CompressTrajectoryResponse = {
185+
memid: "foo",
186+
trajectory: "something",
187+
};
188+
return HttpResponse.json(result);
189+
},
190+
);
191+
192+
export const updateKnowledgeUse: HttpHandler = http.post(
193+
`http://127.0.0.1:8001${KNOWLEDGE_UPDATE_USED_URL}`,
194+
async (ctx) => {
195+
const body = await ctx.request.json();
196+
const memid =
197+
body &&
198+
typeof body === "object" &&
199+
"memid" in body &&
200+
typeof body.memid === "string"
201+
? body.memid
202+
: "foo";
203+
const result: CompressTrajectoryResponse = {
204+
memid,
205+
trajectory: "something",
206+
};
207+
208+
await new Promise((r) => setTimeout(r, 1000));
209+
return HttpResponse.json(result);
210+
},
211+
);
113212
export const loginPollingGood: HttpHandler = http.get(
114213
"https://www.smallcloud.ai/v1/streamlined-login-recall-ticket",
115214
() => {

0 commit comments

Comments
 (0)