Skip to content

Commit bf1d7ef

Browse files
nelly-hatevanelly-hateva
committed
Merge remote-tracking branch 'origin' into Statnett-255
2 parents 227509f + 4b97d30 commit bf1d7ef

File tree

11 files changed

+500
-530
lines changed

11 files changed

+500
-530
lines changed

‎CHANGELOG.md‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,17 @@
1-
Next release
1+
1.4.0-rc1
22
============
33

44
* [#255](https://github.com/statnett/Talk2PowerSystem_PM/issues/255): OBO auth flow for Cognite
5-
* [#255](https://github.com/statnett/Talk2PowerSystem_PM/issues/255): Update the version of `cognite-sdk` from `7.86.0` to `7.89.0`
5+
* [#255](https://github.com/statnett/Talk2PowerSystem_PM/issues/255): Update the version of `cognite-sdk` from `7.88.0` to `7.89.0`
6+
7+
1.3.0-rc1
8+
============
9+
610
* [#256](https://github.com/statnett/Talk2PowerSystem_PM/issues/256): Update the queries for the ontologies and datasets information served from the `__about` endpoint
711
* [#251](https://github.com/statnett/Talk2PowerSystem_PM/issues/251): Change N-Shot tool configuration to default to the base GraphDB
8-
* [#251](https://github.com/statnett/Talk2PowerSystem_PM/issues/251): Change N-Shot tool configuration SPARQL query template
12+
* [#276](https://github.com/statnett/Talk2PowerSystem_PM/issues/276): Change N-Shot tool configuration SPARQL query template, so that it outputs unique SPARQL queries
913
* [#254](https://github.com/statnett/Talk2PowerSystem_PM/issues/254): Update the version of `ttyg` from `1.9.3` to `1.10.0`, so that the chat bot can run without admin access to GraphDB
14+
* [#254](https://github.com/statnett/Talk2PowerSystem_PM/issues/254): Update the version of `cognite-sdk` from `7.86.0` to `7.88.0`
1015

1116
1.2.0-rc4
1217
============

‎README.adoc‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -98,7 +98,8 @@ Steps to create an official release (from the main branch):
9898

9999
. `conda activate Talk2PowerSystemLLM`
100100
. `poetry version <major|minor|patch>`.
101-
. `git add pyproject.toml`
101+
. Update `CHANGELOG.md`
102+
. `git add pyproject.toml CHANGELOG.md`
102103
. `git commit -m "Bumping version from <previous-version> to <current-version>"`
103104
. `git push -u origin main`
104105
. Create a release from https://github.com/statnett/Talk2PowerSystem_LLM/releases[the GitHub interface]. The tag and the release title must match the version from poetry!
@@ -115,7 +116,8 @@ For example, if the current version is `1.2.0-rc1`, the next version must be `1.
115116
If the current version is a development version, then the next pre-release version must follow the semantic versioning convention on how to increment the major,
116117
minor and patch parts of the version and add `-rc1` at the end.
117118
For example, if the current version is `1.1.0-dev0` and the next release will be a major one, the next version must be `2.0.0-rc1`.
118-
. `git add pyproject.toml`
119+
. Update `CHANGELOG.md`
120+
. `git add pyproject.toml CHANGELOG.md`
119121
. `git commit -m "Bumping version from <previous-version> to <current-version>"`
120122
. `git push -u origin main`
121123
. Create a release from https://github.com/statnett/Talk2PowerSystem_LLM/releases[the GitHub interface]. The tag and the release title must match the version from poetry!

‎config/dev+retrieval.yaml‎

Lines changed: 15 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -32,18 +32,22 @@ tools:
3232
PREFIX retr: <http://www.ontotext.com/connectors/retrieval#>
3333
PREFIX retr-index: <http://www.ontotext.com/connectors/retrieval/instance#>
3434
PREFIX qa: <https://www.statnett.no/Talk2PowerSystem/qa#>
35-
SELECT ?question ?query {{
36-
[] a retr-index:{connector_name} ;
37-
retr:query "{query}" ;
38-
retr:limit {limit} ;
39-
retr:entities ?entity .
40-
?entity retr:score ?score;
41-
qa:question ?question.
42-
?template qa:paraphrase ?entity;
43-
qa:querySparql ?query.
44-
FILTER (?score > {score})
35+
SELECT (REPLACE(GROUP_CONCAT(?q; separator="@"), "(.*?)@.*", "$1") AS ?question) ?query {{
36+
SELECT ?q ?query ?score {{
37+
[] a retr-index:{connector_name} ;
38+
retr:query "{query}" ;
39+
retr:limit 100;
40+
retr:entities ?entity .
41+
?entity retr:score ?score;
42+
qa:question ?q.
43+
?template qa:paraphrase ?entity;
44+
qa:querySparql ?query.
45+
FILTER (?score > {score})
46+
}}
47+
ORDER BY DESC(?score)
4548
}}
46-
ORDER BY DESC(?score)
49+
GROUP BY ?query
50+
LIMIT {limit}
4751
llm:
4852
azure_endpoint: "https://statnett.openai.azure.com/"
4953
model: "gpt-4.1"

‎config/ontology/cim-subset-pretty.ttl‎

Lines changed: 449 additions & 399 deletions
Large diffs are not rendered by default.

‎docs/AgentConfig.md‎

Lines changed: 18 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -38,18 +38,22 @@ tools:
3838
PREFIX retr: <http://www.ontotext.com/connectors/retrieval#>
3939
PREFIX retr-index: <http://www.ontotext.com/connectors/retrieval/instance#>
4040
PREFIX qa: <https://www.statnett.no/Talk2PowerSystem/qa#>
41-
SELECT ?question ?query {{
42-
[] a retr-index:{connector_name} ;
43-
retr:query "{query}" ;
44-
retr:limit {limit} ;
45-
retr:entities ?entity .
46-
?entity retr:score ?score;
47-
qa:question ?question.
48-
?template qa:paraphrase ?entity;
49-
qa:querySparql ?query.
50-
FILTER (?score > {score})
41+
SELECT (REPLACE(GROUP_CONCAT(?q; separator="@"), "(.*?)@.*", "$1") AS ?question) ?query {{
42+
SELECT ?q ?query ?score {{
43+
[] a retr-index:{connector_name} ;
44+
retr:query "{query}" ;
45+
retr:limit 100;
46+
retr:entities ?entity .
47+
?entity retr:score ?score;
48+
qa:question ?q.
49+
?template qa:paraphrase ?entity;
50+
qa:querySparql ?query.
51+
FILTER (?score > {score})
52+
}}
53+
ORDER BY DESC(?score)
5154
}}
52-
ORDER BY DESC(?score)
55+
GROUP BY ?query
56+
LIMIT {limit}
5357
cognite:
5458
base_url: https://statnett.cognitedata.com
5559
client_name: talk2powersystem
@@ -146,11 +150,11 @@ LIMIT {limit}
146150
- `tools.cognite.project` - OPTIONAL, DEFAULT=`prod` - Cognite Data Fusion project name.
147151
One of `dev1`, `dev2`, `dev3`, `test`, `prod` according to [CDF access from RNDP](https://github.com/statnett/Talk2PowerSystem_PM/wiki/CDF-access-from-RNDP).
148152
- `tools.cognite.client_name` - OPTIONAL, DEFAULT=`talk2powersystem` - Name of the client for logging purposes.
149-
- `tools.cognite.interactive_client_id` - OPTIONAL - If provided, interactive authentication is used (local run of Jupyter Notebook).
153+
- `tools.cognite.interactive_client_id` - OPTIONAL - If provided, interactive authentication is used (when you run on a dev machine the backend app with uvicorn or the Jupyter Notebook).
150154
Otherwise, `tools.cognite.token_file_path` or `tools.cognite.client_secret` must be provided.
151155
- `tools.cognite.tenant_id` - REQUIRED iff `tools.cognite.interactive_client_id` is present - Azure tenant ID. For example, `a8d61462-f252-44b2-bf6a-d7231960c041`.
152-
- `tools.cognite.token_file_path` - OPTIONAL - Full path on the disk to the cognite token file (run of Jupyter Notebook on RNDP). For example, `/var/run/secrets/microsoft.com/entra/cognite`.
153-
* `tools.cognite.client_secret` - OPTIONAL - Client secret for the Cognite confidential application (running from the backend app).
156+
- `tools.cognite.token_file_path` - OPTIONAL - Full path on the disk to the cognite token file (used when you run the Jupyter Notebook on RNDP). For example, `/var/run/secrets/microsoft.com/entra/cognite`.
157+
* `tools.cognite.client_secret` - OPTIONAL - Client secret for the Cognite confidential application (used for the backend app running on RNDP).
154158
155159
## `llm`
156160

‎pyproject.toml‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[project]
22
name = "Talk2PowerSystemLLM"
3-
version = "1.2.0-rc4"
3+
version = "1.4.0-rc1"
44
description = "Talk to Power System LLM"
55
authors = []
66
readme = "README.adoc"
@@ -51,5 +51,4 @@ build-backend = "poetry.core.masonry.api"
5151
[project.scripts]
5252
evaluation = 'talk2powersystemllm.scripts.run_evaluation:main'
5353
qa_dataset2rdf = 'talk2powersystemllm.scripts.qa_dataset2rdf:main'
54-
find_min_retrieval_limit_on_dev = 'talk2powersystemllm.scripts.find_min_retrieval_limit_on_dev:main'
5554
benchmark_graphdb_ttyg = 'talk2powersystemllm.scripts.benchmark_graphdb_ttyg:main'

‎src/jupyter_notebooks/Talk2PowerSystem.ipynb‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -111,7 +111,7 @@
111111
"\n",
112112
"\n",
113113
"conf = RunnableConfig(configurable={\"thread_id\": \"thread-123\"})\n",
114-
"messages = {\"messages\": [(\"user\", \"List timeseries\")]}\n",
114+
"messages = {\"messages\": [(\"user\", \"List all transformers within substation OSLO.\")]}\n",
115115
"last_message_id = run_agent(agent, messages, conf)"
116116
]
117117
},

‎src/talk2powersystemllm/app/trouble.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1345,7 +1345,7 @@ Sample Response Body:
13451345
},
13461346
"sample_sparql_queries": {
13471347
"enabled": true,
1348-
"sparql_query_template": "PREFIX retr: <http://www.ontotext.com/connectors/retrieval#>\nPREFIX retr-index: <http://www.ontotext.com/connectors/retrieval/instance#>\nPREFIX qa: <https://www.statnett.no/Talk2PowerSystem/qa#>\nSELECT ?question ?query {{\n [] a retr-index:{connector_name} ;\n retr:query \"{query}\" ;\n retr:limit {limit} ;\n retr:entities ?entity .\n ?entity retr:score ?score;\n qa:question ?question.\n ?template qa:paraphrase ?entity;\n qa:querySparql ?query.\n FILTER (?score > {score})\n}}\nORDER BY DESC(?score)\n",
1348+
"sparql_query_template": "PREFIX retr: <http://www.ontotext.com/connectors/retrieval#>\nPREFIX retr-index: <http://www.ontotext.com/connectors/retrieval/instance#>\nPREFIX qa: <https://www.statnett.no/Talk2PowerSystem/qa#>\nSELECT (REPLACE(GROUP_CONCAT(?q; separator=\"@\"), \"(.*?)@.*\", \"$1\") AS ?question) ?query {{\n SELECT ?q ?query ?score {{\n [] a retr-index:{connector_name} ;\n retr:query \"{query}\" ;\n retr:limit 100;\n retr:entities ?entity .\n ?entity retr:score ?score;\n qa:question ?q.\n ?template qa:paraphrase ?entity;\n qa:querySparql ?query.\n FILTER (?score > {score})\n }}\n ORDER BY DESC(?score)\n}}\nGROUP BY ?query\nLIMIT {limit}\n",
13491349
"connector_name": "qa_dataset"
13501350
},
13511351
"retrieve_data_points": {

‎src/talk2powersystemllm/qa_dataset/min_retrieval_limit.py‎

Lines changed: 0 additions & 48 deletions
This file was deleted.

‎src/talk2powersystemllm/qa_dataset/qa_dataset2rdf.py‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -64,7 +64,8 @@ def build_qa_dataset_graph(split):
6464
paraphrase_iri = URIRef(f"Paraphrase_{template['template_id']}_{n}", base_ns)
6565
graph.add((paraphrase_iri, RDF.type, qa_dataset_ns.Paraphrase))
6666
graph.add((template_iri, qa_dataset_ns.paraphrase, paraphrase_iri))
67-
graph.add((paraphrase_iri, qa_dataset_ns.question, Literal(transform_paraphrase(paraphrase))))
68-
verify_unique_placeholders(transform_paraphrase(paraphrase))
67+
transformed_paraphrase = transform_paraphrase(paraphrase)
68+
verify_unique_placeholders(transformed_paraphrase)
69+
graph.add((paraphrase_iri, qa_dataset_ns.question, Literal(transformed_paraphrase)))
6970

7071
return graph

0 commit comments

Comments
 (0)