Implemented support for sections in Plain.

Author: dusano

Plain-language-specification.md

@@ -13,8 +13,6 @@ Plain language is structured English based on markdown syntax.
 Here's an example of a "hello,world" program in Plain.
 
 ```plain
-***Definitions:***
-
 ***Non-Functional Requirements:***
 
 - Implementation should be in Python.
@@ -26,19 +24,42 @@ Here's an example of a "hello,world" program in Plain.
 
 # Source structure
 
-Every Plain source file requires the following top level sections:
+## Source organization
+
+Plain source can be organized in sections and subsection using markdown headers.
+
+```plain
+# Section 1
+
+# Section 2
+
+***Non-Functional Requirements:***
+
+- Simple non-functional requirement
+
+## Section 2.1
+
+***Functional Requirements:***
+
+- Simple functional requirement
+```
+
+### Specifications
+
+There are four types of specifications:
 
 - `***Definitions:***`
 - `***Non-Functional Requirements:***`
 - `***Functional Requirements:***`
-
-The following top level section is optional:
 - `***Test Requirements:***`
 
+Every plain source file requires at least one functional requirement and an associated non-functional requirement.
+
+Functional requirements must reside in leaf sections while other specifications can be placed also in non-leaf sections. Specifications in non-leaf sections apply not just to the section itself but to all of its subsections.
 
-## Definitons
+## Definitions
 
-The top level section `***Definitions:***` is a list of definitions of new terms.
+The `***Definitions:***` specification is a list of definitions of new terms.
 
 Here's an example of a simple definiton.
 
@@ -61,7 +82,7 @@ The definition of a term is provided in natural language. There are no restricti
 
 ## Non-Functional Requirements
 
-The top level section `***Non-Functional Requirements:***` is a list of instructions that steer software code implementation and provide details of execution environment.
+The `***Non-Functional Requirements:***` specification is a list of instructions that steer software code implementation and provide details of execution environment.
 
 Here's an example of a simple instruction specifying only that the Plain specification should be rendered to Python software code.
 
@@ -83,7 +104,7 @@ Here's an example of more complex instructions.
 
 ## Functional Requirements
 
-The top level section `***Functional Requirements:***` provides a description of functionality that should be rendered to software code. The descriptions should be provided in natural language as a markdown list. When referring to other terms **The Noun convention** should be followed.
+The `***Functional Requirements:***` specificationprovides a description of functionality that should be rendered to software code. The descriptions should be provided in natural language as a markdown list. When referring to other terms **The Noun convention** should be followed.
 
 Here's an example of a simple description of the functionality of the "hello, world" application.
 
@@ -125,7 +146,7 @@ Functional requirements are rendered incrementally one by one. Consequently earl
 
 ## Test Requirements
 
-The top level section `***Test Requirements:***` is a list of instructions that steer implementation of end-to-end tests and provide details of testing environment.
+The `***Test Requirements:***` specification is a list of instructions that steer implementation of end-to-end tests and provide details of testing environment.
 
 Here's an example specification of test requirements.
 

codeplain_REST_api.py

@@ -57,42 +57,21 @@ def post_request(self, endpoint_url, headers, payload):
         return response_json
 
 
-    def get_plain_sections(self, plain_source, loaded_templates):
+    def get_plain_source_tree(self, plain_source, loaded_templates):
         """
-        Extracts labeled sections from the given plain text source in Markdown format.
+        Builds plain source tree from the given plain text source in Markdown format.
 
         Args:
             plain_source (str): A string containing the plain text source to be parsed. 
-                                The input must be in Markdown format and include the following 
-                                mandatory labeled sections, each marked by a specific label 
-                                at the start of the line:
-                                
-                                - "***Definitions:***"
-                                - "***Functional Requirements:***"
-                                - "***Non-Functional Requirements:***"
-                                
-                                Other labeled sections, such as "***Test Requirements:***", 
-                                are optional but will be parsed if present. The input should be 
-                                non-empty and formatted using consistent Markdown syntax with 
-                                the required labels to ensure successful parsing and extraction.
             loaded_templates (dict): A dictionary containing the loaded templates.
 
         Returns:
-            list: A list of labeled sections extracted from the plain source. Each section is 
-                a structured segment within the source text, starting with a recognized 
-                labeled header followed by its content.
+            dict: A plain source tree.
 
         Raises:
             Exception: If parsing of plain_source fails.
-
-        Notes:
-            This method processes the input text using predefined parsing rules to identify 
-            and extract labeled sections. The document must be formatted correctly in Markdown, 
-            with specific labels (e.g., "***Definitions:***") marking the start of each segment. 
-            The "Definitions," "Functional Requirements," and "Non-Functional Requirements" 
-            labeled sections are required, while others are optional.
         """
-        endpoint_url = f"{self.api_url}/plain_sections"
+        endpoint_url = f"{self.api_url}/plain_source_tree"
         headers = {
             "X-API-Key": self.api_key,
             "Content-Type": "application/json"
@@ -106,15 +85,14 @@ def get_plain_sections(self, plain_source, loaded_templates):
         return self.post_request(endpoint_url, headers, payload)
 
 
-    def render_functional_requirement(self, frid, plain_sections, linked_resources, existing_files_content):
+    def render_functional_requirement(self, frid, plain_source_tree, linked_resources, existing_files_content):
         """
         Renders the content of a functional requirement based on the provided ID, 
-        corresponding sections from a Plain document, and existing files' content.
+        plain source tree, and existing files' content.
 
         Args:
             frid (str): The unique identifier for the functional requirement to be rendered.
-            plain_sections (list of str): A list of sections from the Plain document that 
-                                        are relevant to the functional requirement.
+            plain_source_tree (dict): A dictionary containing the plain source tree.
             linked_resources (dict): A dictionary where the keys represent resource names 
                                     and the values are the content of those resources.
             existing_files_content (dict): A dictionary where the keys represent code base
@@ -125,7 +103,7 @@ def render_functional_requirement(self, frid, plain_sections, linked_resources,
                 appropriately based on the inputs.
 
         Raises:
-            ValueError: If the frid is invalid or the necessary sections cannot be found.
+            ValueError: If the frid is invalid or the necessary plain source tree is not valid.
         """
         endpoint_url = f"{self.api_url}/render_functional_requirement"
         headers = {
@@ -135,15 +113,15 @@ def render_functional_requirement(self, frid, plain_sections, linked_resources,
 
         payload = {
             "frid": frid,
-            "plain_sections": plain_sections,
+            "plain_source_tree": plain_source_tree,
             "linked_resources": linked_resources,
             "existing_files_content": existing_files_content
         }
-        
+
         return self.post_request(endpoint_url, headers, payload)
 
 
-    def fix_unittests_issue(self, frid, plain_sections, linked_resources, existing_files_content, unittests_issue):
+    def fix_unittests_issue(self, frid, plain_source_tree, linked_resources, existing_files_content, unittests_issue):
         endpoint_url = f"{self.api_url}/fix_unittests_issue"
         headers = {
             "X-API-Key": self.api_key,
@@ -152,7 +130,7 @@ def fix_unittests_issue(self, frid, plain_sections, linked_resources, existing_f
 
         payload = {
             "frid": frid,
-            "plain_sections": plain_sections,
+            "plain_source_tree": plain_source_tree,
             "linked_resources": linked_resources,
             "existing_files_content": existing_files_content,
             "unittests_issue": unittests_issue
@@ -176,7 +154,7 @@ def refactor_source_files_if_needed(self, files_to_check, existing_files_content
         return self.post_request(endpoint_url, headers, payload)
 
 
-    def render_e2e_tests(self, frid, plain_sections, linked_resources, existing_files_content):
+    def render_e2e_tests(self, frid, plain_source_tree, linked_resources, existing_files_content):
         endpoint_url = f"{self.api_url}/render_e2e_tests"
         headers = {
             "X-API-Key": self.api_key,
@@ -185,7 +163,7 @@ def render_e2e_tests(self, frid, plain_sections, linked_resources, existing_file
 
         payload = {
             "frid": frid,
-            "plain_sections": plain_sections,
+            "plain_source_tree": plain_source_tree,
             "linked_resources": linked_resources,
             "existing_files_content": existing_files_content
         }
@@ -208,7 +186,7 @@ def generate_folder_name_from_functional_requirement(self, functional_requiremen
         return self.post_request(endpoint_url, headers, payload)
 
 
-    def fix_e2e_tests_issue(self, frid, functional_requirement_id, plain_sections, linked_resources, existing_files_content, code_diff, e2e_tests_files, e2e_tests_issue, implementation_fix_count):
+    def fix_e2e_tests_issue(self, frid, functional_requirement_id, plain_source_tree, linked_resources, existing_files_content, code_diff, e2e_tests_files, e2e_tests_issue, implementation_fix_count):
         endpoint_url = f"{self.api_url}/fix_e2e_tests_issue"
         headers = {
             "X-API-Key": self.api_key,
@@ -218,7 +196,7 @@ def fix_e2e_tests_issue(self, frid, functional_requirement_id, plain_sections, l
         payload = {
             "frid": frid,
             "functional_requirement_id": functional_requirement_id,
-            "plain_sections": plain_sections,
+            "plain_source_tree": plain_source_tree,
             "linked_resources": linked_resources,
             "existing_files_content": existing_files_content,
             "code_diff": code_diff,