new efficient growth in pool efficient taking into account idle connections.

Author: TimelordUK

docs/pool-efficient-strategy.md

@@ -0,0 +1,102 @@
+# Pool Efficient Strategy
+
+## Overview
+
+The `efficient` scaling strategy provides optimized pool growth behavior that only creates new connections when existing idle connections are exhausted. This addresses the inefficiency in the default `aggressive` strategy where the pool grows to the ceiling immediately on open.
+
+## Problem Solved
+
+The original pool behavior had these inefficiencies:
+
+1. **Eager growth**: With `aggressive` strategy, pools grew to ceiling immediately on open
+2. **Unnecessary connections**: Created connections even when idle ones were available
+3. **Resource waste**: Maintained more connections than needed for typical workloads
+
+## Solution: Efficient Strategy
+
+### Behavior
+
+- **Initial creation**: Creates connections only up to `floor` on pool open
+- **Conservative growth**: Only grows when work queue exceeds idle connections
+- **Incremental scaling**: Adds connections based on actual demand
+
+### Usage
+
+```javascript
+const pool = new sql.Pool({
+  connectionString: '...',
+  floor: 5,           // Start with 5 connections
+  ceiling: 20,        // Maximum 20 connections  
+  scalingStrategy: 'efficient',    // Use efficient strategy
+  scalingIncrement: 3              // Grow by 3 when needed
+})
+```
+
+### Comparison of Strategies
+
+| Strategy | Initial Growth | Additional Growth | Use Case |
+|----------|----------------|------------------|----------|
+| `aggressive` | To ceiling immediately | None needed | High constant load |
+| `gradual` | To ceiling incrementally | Fixed increments | Predictable growth |
+| `exponential` | To floor, then exponential | By growth factor | Adaptive scaling |
+| `efficient` | To floor only | Only when needed | Variable/bursty load |
+
+## Examples
+
+### Scenario 1: Burst Workload
+```javascript
+// With efficient strategy
+const pool = new sql.Pool({
+  floor: 5,
+  ceiling: 20,
+  scalingStrategy: 'efficient'
+})
+
+// 1. Pool starts with 5 connections
+// 2. Submit 5 queries → uses existing connections, no growth
+// 3. Submit 10 queries → grows only when idle connections exhausted
+// 4. Later: idle connections reused for new work
+```
+
+### Scenario 2: Variable Load
+```javascript
+// Perfect for applications with:
+// - Variable query load
+// - Periods of low activity  
+// - Need to minimize connection overhead
+// - Want to reuse existing connections efficiently
+```
+
+## Benefits
+
+1. **Resource efficiency**: Uses fewer connections for typical workloads
+2. **Better connection reuse**: Maximizes use of existing idle connections
+3. **Reduced overhead**: Lower memory and connection overhead
+4. **Backward compatible**: Other strategies remain unchanged
+
+## Migration
+
+Existing code continues to work unchanged. To opt into efficient behavior:
+
+```javascript
+// Old (aggressive by default)
+const pool = new sql.Pool({ connectionString, ceiling: 20 })
+
+// New (efficient)  
+const pool = new sql.Pool({ 
+  connectionString, 
+  floor: 5,
+  ceiling: 20, 
+  scalingStrategy: 'efficient' 
+})
+```
+
+## Testing
+
+The efficient strategy is thoroughly tested to ensure:
+- Only grows when idle connections are exhausted
+- Respects floor and ceiling limits
+- Maintains backward compatibility
+- Handles burst workloads correctly
+
+See `test/pool-efficient-strategy.test.js` for comprehensive test coverage.

lib/index.d.ts

@@ -187,8 +187,9 @@ declare namespace MsNodeSqlV8 {
      * - 'aggressive': immediately create all needed connections (default)
      * - 'gradual': create connections in fixed increments
      * - 'exponential': create connections based on exponential growth factor
+     * - 'efficient': only grow when idle connections are exhausted
      */
-    scalingStrategy?: 'aggressive' | 'gradual' | 'exponential'
+    scalingStrategy?: 'aggressive' | 'gradual' | 'exponential' | 'efficient'
 
     /**
      * number of connections to create at once when using 'gradual' strategy

lib/pool.js

@@ -434,28 +434,49 @@ const poolModule = (() => {
           return
         }
 
-        // First, promote any unpaused items from pause queue
-        promotePause()
-        
-        // Process work with existing idle connections
-        while (workQueue.length > 0 && idle.length > 0) {
-          const work = workQueue.pop()
-          if (work.poolNotifier.isPendingCancel()) {
-            _this.emit('debug', `query work id = ${work.id} has been cancelled waiting in pool to execute, workQueue = ${workQueue.length}`)
-            doneFree(work.poolNotifier)
-          } else if (work.poolNotifier.isPaused()) {
-            pause.unshift(work)
-          } else {
-            const description = checkout('work')
-            item(description, work)
+        // If using 'efficient' strategy, only grow when work queue exceeds idle connections
+        if (options.scalingStrategy === 'efficient') {
+          // First process work with existing idle connections
+          promotePause()
+          while (workQueue.length > 0 && idle.length > 0) {
+            const work = workQueue.pop()
+            if (work.poolNotifier.isPendingCancel()) {
+              _this.emit('debug', `query work id = ${work.id} has been cancelled waiting in pool to execute, workQueue = ${workQueue.length}`)
+              doneFree(work.poolNotifier)
+            } else if (work.poolNotifier.isPaused()) {
+              pause.unshift(work)
+            } else {
+              const description = checkout('work')
+              item(description, work)
+            }
           }
-        }
-        
-        // Only grow the pool if we still have work queued and no idle connections
-        if (workQueue.length > 0 && idle.length === 0) {
-          _this.emit('debug', `crank needs to grow pool: workQueue = ${workQueue.length}, idle = ${idle.length}, busy = ${busyConnectionCount}`)
+          
+          // Only grow if we still have work queued and no idle connections
+          if (workQueue.length > 0 && idle.length === 0) {
+            const existing = idle.length + busyConnectionCount + pendingCreates + parkingConnectionCount
+            if (existing < options.ceiling) {
+              _this.emit('debug', `efficient crank growing: workQueue = ${workQueue.length}, idle = ${idle.length}, busy = ${busyConnectionCount}`)
+              void grow().then(() => {
+                // Process any remaining work after growth
+                promotePause()
+                while (workQueue.length > 0 && idle.length > 0) {
+                  const work = workQueue.pop()
+                  if (work.poolNotifier.isPendingCancel()) {
+                    _this.emit('debug', `query work id = ${work.id} has been cancelled waiting in pool to execute, workQueue = ${workQueue.length}`)
+                    doneFree(work.poolNotifier)
+                  } else if (work.poolNotifier.isPaused()) {
+                    pause.unshift(work)
+                  } else {
+                    const description = checkout('work')
+                    item(description, work)
+                  }
+                }
+              })
+            }
+          }
+        } else {
+          // Original behavior for backward compatibility
           void grow().then(() => {
-            // After growing, process any remaining work
             promotePause()
             while (workQueue.length > 0 && idle.length > 0) {
               const work = workQueue.pop()
@@ -664,10 +685,11 @@ const poolModule = (() => {
           return
         }
 
-        // For initial pool creation, grow to floor if not yet reached
-        const targetSize = opened ? options.ceiling : options.floor
-        if (existing >= targetSize) {
-          return
+        // For efficient strategy, limit initial growth to floor unless there's work waiting
+        if (options.scalingStrategy === 'efficient' && !opened) {
+          if (existing >= options.floor && workQueue.length === 0) {
+            return
+          }
         }
 
         function connectionOptions (c) {
@@ -685,9 +707,24 @@ const poolModule = (() => {
 
         // Calculate how many connections to create based on strategy
         let connectionsToCreate = 0
-        const needed = targetSize - existing
+        const needed = options.ceiling - existing
 
         switch (options.scalingStrategy) {
+          case 'efficient': {
+            // For efficient strategy, create conservatively
+            if (!opened) {
+              // During initial open, only create up to floor
+              connectionsToCreate = Math.min(options.floor - existing, needed)
+            } else {
+              // After opening, grow incrementally based on work queue
+              const queuedWork = workQueue.length
+              const additionalNeeded = Math.max(0, queuedWork - idle.length)
+              connectionsToCreate = Math.min(additionalNeeded, needed, options.scalingIncrement || 2)
+            }
+            connectionsToCreate = Math.max(0, connectionsToCreate)
+            break
+          }
+
           case 'gradual':
           // Create a fixed increment of connections
             connectionsToCreate = Math.min(options.scalingIncrement, needed)