**docs(guides): add service identification guide**

- Created `service-identification.md` in `docs/guides`:
  - Detailed guidelines on how services should identify themselves within Brainy.
  - Documented two identification methods: default service initialization and operation-specific service naming.
  - Included service name conventions and common examples (`github`, `reddit`, `default`).
- Described benefits of proper service identification:
  - Enhanced statistics tracking and JSON field discovery by service.
- Provided best practices for consistent and descriptive service naming.
- Explained internal implementation details, such as `getServiceName` usage and statistic tracking.

**Purpose**: Help users properly identify services to enable statistics tracking, field discovery, and improved data management in Brainy.

docs/guides/service-identification.md

@@ -0,0 +1,151 @@
+# Service Identification Guide
+
+## Overview
+
+This guide explains how services should identify themselves when adding, updating, or deleting data in Brainy. Proper service identification is important for:
+
+1. **Statistics tracking**: Brainy tracks statistics by service, allowing you to see how much data each service has contributed.
+2. **JSON field discovery**: Brainy tracks field names in JSON documents by service, allowing you to discover what fields are available from different services.
+
+## How to Identify Your Service
+
+### Option 1: Specify a Default Service During Initialization
+
+The simplest approach is to specify a default service name when initializing Brainy:
+
+```javascript
+// Initialize Brainy with a default service name
+const brainyData = new BrainyData({
+  defaultService: 'my-service-name'
+});
+
+// All operations will automatically use the default service name
+await brainyData.add(data, metadata);
+await brainyData.updateMetadata(id, newMetadata);
+await brainyData.delete(id);
+```
+
+This approach ensures that all operations use the same service name without having to specify it for each operation.
+
+### Option 2: Specify Service Name for Each Operation
+
+Alternatively, you can specify the service name for each operation:
+
+```javascript
+// Adding data with service identification
+const id = await brainyData.add(data, metadata, { 
+  service: 'my-service-name' 
+});
+
+// Updating metadata with service identification
+await brainyData.updateMetadata(id, newMetadata, {
+  service: 'my-service-name'
+});
+
+// Deleting data with service identification
+await brainyData.delete(id, {
+  service: 'my-service-name'
+});
+```
+
+This approach gives you more flexibility if you need to use different service names for different operations.
+
+## Service Name Conventions
+
+Service names should be:
+
+- Lowercase
+- Hyphen-separated (e.g., `github-api`, `bluesky-feed`)
+- Descriptive of the data source or service
+- Consistent across all operations from the same service
+
+Common service names used in Brainy include:
+
+- `github`
+- `bluesky`
+- `reddit`
+- `twitter`
+- `google`
+- `default` (used when no service is specified)
+
+## How Service Identification Works
+
+Internally, Brainy uses the `getServiceName` helper method to determine the service name:
+
+1. If a `service` parameter is provided in the options object, that value is used.
+2. If no `service` parameter is provided, Brainy uses the default service name specified during initialization.
+3. If no default service was specified, `'default'` is used.
+
+## Benefits of Proper Service Identification
+
+### Statistics Tracking
+
+With proper service identification, you can see statistics broken down by service:
+
+```javascript
+const stats = await brainyData.getStatistics();
+console.log('Noun count by service:');
+console.log(stats.nounCount);
+console.log('Metadata count by service:');
+console.log(stats.metadataCount);
+```
+
+Example output:
+```
+Noun count by service:
+{
+  "github": 1250,
+  "bluesky": 843,
+  "reddit": 567,
+  "default": 122
+}
+```
+
+### Field Name Discovery
+
+You can discover what fields are available from different services:
+
+```javascript
+const fieldNames = await brainyData.getAvailableFieldNames();
+console.log(fieldNames);
+```
+
+Example output:
+```
+{
+  "github": ["repository.name", "repository.description", "user.login", "issue.title", ...],
+  "bluesky": ["post.text", "user.handle", "user.displayName", ...],
+  "reddit": ["title", "selftext", "author.name", "subreddit.name", ...]
+}
+```
+
+### Standard Field Mappings
+
+You can use standard field mappings to search consistently across different services:
+
+```javascript
+// Get standard field mappings
+const standardFieldMappings = await brainyData.getStandardFieldMappings();
+console.log(standardFieldMappings);
+
+// Search using standard fields
+const results = await brainyData.searchByStandardField("title", "climate change", 10);
+```
+
+## Best Practices
+
+1. **Specify a default service name during initialization** for simplicity and consistency.
+2. **Override the default service name** only when necessary for specific operations.
+3. **Use consistent service names** across all operations from the same service.
+4. **Use descriptive service names** that identify the data source.
+5. **Document the field names** used by your service to help users understand what fields are available for searching.
+
+## Implementation Details
+
+Service identification is implemented through:
+
+1. The `getServiceName` helper method in `BrainyData` class
+2. The `trackFieldNames` method in `StorageAdapter` interface
+3. The `incrementStatistic` and `decrementStatistic` methods in `StorageAdapter` interface
+
+These methods ensure that statistics and field names are properly tracked by service.

src/brainyData.ts

@@ -55,6 +55,13 @@ export interface BrainyDataConfig {
    */
   hnsw?: Partial<HNSWOptimizedConfig>
 
+  /**
+   * Default service name to use for all operations
+   * When specified, this service name will be used for all operations
+   * that don't explicitly provide a service name
+   */
+  defaultService?: string
+
   /**
    * Distance function to use for similarity calculations
    */
@@ -344,6 +351,7 @@ export class BrainyData<T = any> implements BrainyDataInterface<T> {
   private useOptimizedIndex: boolean = false
   private _dimensions: number
   private loggingConfig: BrainyDataConfig['logging'] = { verbose: true }
+  private defaultService: string = 'default'
 
   // Timeout and retry configuration
   private timeoutConfig: BrainyDataConfig['timeouts'] = {}
@@ -455,6 +463,11 @@ export class BrainyData<T = any> implements BrainyDataInterface<T> {
       throw new Error('Database cannot be both read-only and write-only')
     }
 
+    // Set default service name if provided
+    if (config.defaultService) {
+      this.defaultService = config.defaultService
+    }
+
     // Store storage configuration for later use in init()
     this.storageConfig = config.storage || {}
 
@@ -873,7 +886,7 @@ export class BrainyData<T = any> implements BrainyDataInterface<T> {
   }
 
   /**
-   * Get the service name from options or fallback to current augmentation
+   * Get the service name from options or fallback to default service
    * This provides a consistent way to handle service names across all methods
    * @param options Options object that may contain a service property
    * @returns The service name to use for operations
@@ -882,7 +895,9 @@ export class BrainyData<T = any> implements BrainyDataInterface<T> {
     if (options?.service) {
       return options.service
     }
-    return this.getCurrentAugmentation()
+    // Use the default service name specified during initialization
+    // This simplifies service identification by allowing it to be specified once
+    return this.defaultService
   }
 
   /**
@@ -2377,7 +2392,7 @@ export class BrainyData<T = any> implements BrainyDataInterface<T> {
         }
 
         // Get the service that's updating the metadata
-        const service = options.service || this.getCurrentAugmentation()
+        const service = this.getServiceName(options)
         const graphNoun = metadata as unknown as GraphNoun
 
         // Preserve existing createdBy and createdAt if they exist
@@ -2424,7 +2439,7 @@ export class BrainyData<T = any> implements BrainyDataInterface<T> {
       await this.storage!.saveMetadata(id, metadata)
 
       // Track metadata statistics
-      const service = options.service || this.getCurrentAugmentation()
+      const service = this.getServiceName(options)
       await this.storage!.incrementStatistic('metadata', service)
 
       return true
@@ -2534,7 +2549,7 @@ export class BrainyData<T = any> implements BrainyDataInterface<T> {
           const placeholderVector = new Array(this._dimensions).fill(0)
 
           // Add metadata if provided
-          const service = options.service || this.getCurrentAugmentation()
+          const service = this.getServiceName(options)
           const now = new Date()
           const timestamp = {
             seconds: Math.floor(now.getTime() / 1000),
@@ -2576,7 +2591,7 @@ export class BrainyData<T = any> implements BrainyDataInterface<T> {
           const placeholderVector = new Array(this._dimensions).fill(0)
 
           // Add metadata if provided
-          const service = options.service || this.getCurrentAugmentation()
+          const service = this.getServiceName(options)
           const now = new Date()
           const timestamp = {
             seconds: Math.floor(now.getTime() / 1000),
@@ -2685,7 +2700,7 @@ export class BrainyData<T = any> implements BrainyDataInterface<T> {
       // Note: We're no longer validating against VerbType enum to allow custom relationship types
 
       // Get service name from options or current augmentation
-      const service = options.service || this.getCurrentAugmentation()
+      const service = this.getServiceName(options)
 
       // Create timestamp for creation/update time
       const now = new Date()
@@ -3271,6 +3286,78 @@ export class BrainyData<T = any> implements BrainyDataInterface<T> {
     }
   }
 
+  /**
+   * Calculate similarity between two vectors or between two pieces of text/data
+   * This method allows clients to directly calculate similarity scores between items
+   * without needing to add them to the database
+   *
+   * @param a First vector or text/data to compare
+   * @param b Second vector or text/data to compare
+   * @param options Additional options
+   * @returns A promise that resolves to the similarity score (higher means more similar)
+   */
+  public async calculateSimilarity(
+    a: Vector | string | string[],
+    b: Vector | string | string[],
+    options: {
+      forceEmbed?: boolean // Force using the embedding function even if input is a vector
+      distanceFunction?: DistanceFunction // Optional custom distance function
+    } = {}
+  ): Promise<number> {
+    await this.ensureInitialized()
+
+    try {
+      // Convert inputs to vectors if needed
+      let vectorA: Vector
+      let vectorB: Vector
+
+      // Process first input
+      if (
+        Array.isArray(a) &&
+        a.every((item) => typeof item === 'number') &&
+        !options.forceEmbed
+      ) {
+        // Input is already a vector
+        vectorA = a
+      } else {
+        // Input needs to be vectorized
+        try {
+          vectorA = await this.embeddingFunction(a)
+        } catch (embedError) {
+          throw new Error(`Failed to vectorize first input: ${embedError}`)
+        }
+      }
+
+      // Process second input
+      if (
+        Array.isArray(b) &&
+        b.every((item) => typeof item === 'number') &&
+        !options.forceEmbed
+      ) {
+        // Input is already a vector
+        vectorB = b
+      } else {
+        // Input needs to be vectorized
+        try {
+          vectorB = await this.embeddingFunction(b)
+        } catch (embedError) {
+          throw new Error(`Failed to vectorize second input: ${embedError}`)
+        }
+      }
+
+      // Calculate distance using the specified or default distance function
+      const distanceFunction = options.distanceFunction || this.distanceFunction
+      const distance = distanceFunction(vectorA, vectorB)
+
+      // Convert distance to similarity score (1 - distance for cosine)
+      // Higher value means more similar
+      return 1 - distance
+    } catch (error) {
+      console.error('Failed to calculate similarity:', error)
+      throw new Error(`Failed to calculate similarity: ${error}`)
+    }
+  }
+
   /**
    * Search for verbs by type and/or vector similarity
    * @param queryVectorOrData Query vector or data to search for