**docs(guides): add service identification guide**

- Created `service-identification.md` in `docs/guides`: - Detailed guidelines on how services should identify themselves within Brainy. - Documented two identification methods: default service initialization and operation-specific service naming. - Included service name conventions and common examples (`github`, `reddit`, `default`). - Described benefits of proper service identification: - Enhanced statistics tracking and JSON field discovery by service. - Provided best practices for consistent and descriptive service naming. - Explained internal implementation details, such as `getServiceName` usage and statistic tracking. **Purpose**: Help users properly identify services to enable statistics tracking, field discovery, and improved data management in Brainy.
2025-08-01 10:16:18 -07:00 · 2025-08-01 10:16:18 -07:00 · 6ee0881d86
commit 6ee0881d86
parent d05d381a5d
2 changed files with 245 additions and 7 deletions
--- a/docs/guides/service-identification.md
+++ b/docs/guides/service-identification.md
@ -0,0 +1,151 @@
+# Service Identification Guide
+
+## Overview
+
+This guide explains how services should identify themselves when adding, updating, or deleting data in Brainy. Proper service identification is important for:
+
+1. **Statistics tracking**: Brainy tracks statistics by service, allowing you to see how much data each service has contributed.
+2. **JSON field discovery**: Brainy tracks field names in JSON documents by service, allowing you to discover what fields are available from different services.
+
+## How to Identify Your Service
+
+### Option 1: Specify a Default Service During Initialization
+
+The simplest approach is to specify a default service name when initializing Brainy:
+
+```javascript
+// Initialize Brainy with a default service name
+const brainyData = new BrainyData({
+  defaultService: 'my-service-name'
+});
+
+// All operations will automatically use the default service name
+await brainyData.add(data, metadata);
+await brainyData.updateMetadata(id, newMetadata);
+await brainyData.delete(id);
+```
+
+This approach ensures that all operations use the same service name without having to specify it for each operation.
+
+### Option 2: Specify Service Name for Each Operation
+
+Alternatively, you can specify the service name for each operation:
+
+```javascript
+// Adding data with service identification
+const id = await brainyData.add(data, metadata, { 
+  service: 'my-service-name' 
+});
+
+// Updating metadata with service identification
+await brainyData.updateMetadata(id, newMetadata, {
+  service: 'my-service-name'
+});
+
+// Deleting data with service identification
+await brainyData.delete(id, {
+  service: 'my-service-name'
+});
+```
+
+This approach gives you more flexibility if you need to use different service names for different operations.
+
+## Service Name Conventions
+
+Service names should be:
+
+- Lowercase
+- Hyphen-separated (e.g., `github-api`, `bluesky-feed`)
+- Descriptive of the data source or service
+- Consistent across all operations from the same service
+
+Common service names used in Brainy include:
+
+- `github`
+- `bluesky`
+- `reddit`
+- `twitter`
+- `google`
+- `default` (used when no service is specified)
+
+## How Service Identification Works
+
+Internally, Brainy uses the `getServiceName` helper method to determine the service name:
+
+1. If a `service` parameter is provided in the options object, that value is used.
+2. If no `service` parameter is provided, Brainy uses the default service name specified during initialization.
+3. If no default service was specified, `'default'` is used.
+
+## Benefits of Proper Service Identification
+
+### Statistics Tracking
+
+With proper service identification, you can see statistics broken down by service:
+
+```javascript
+const stats = await brainyData.getStatistics();
+console.log('Noun count by service:');
+console.log(stats.nounCount);
+console.log('Metadata count by service:');
+console.log(stats.metadataCount);
+```
+
+Example output:
+```
+Noun count by service:
+{
+  "github": 1250,
+  "bluesky": 843,
+  "reddit": 567,
+  "default": 122
+}
+```
+
+### Field Name Discovery
+
+You can discover what fields are available from different services:
+
+```javascript
+const fieldNames = await brainyData.getAvailableFieldNames();
+console.log(fieldNames);
+```
+
+Example output:
+```
+{
+  "github": ["repository.name", "repository.description", "user.login", "issue.title", ...],
+  "bluesky": ["post.text", "user.handle", "user.displayName", ...],
+  "reddit": ["title", "selftext", "author.name", "subreddit.name", ...]
+}
+```
+
+### Standard Field Mappings
+
+You can use standard field mappings to search consistently across different services:
+
+```javascript
+// Get standard field mappings
+const standardFieldMappings = await brainyData.getStandardFieldMappings();
+console.log(standardFieldMappings);
+
+// Search using standard fields
+const results = await brainyData.searchByStandardField("title", "climate change", 10);
+```
+
+## Best Practices
+
+1. **Specify a default service name during initialization** for simplicity and consistency.
+2. **Override the default service name** only when necessary for specific operations.
+3. **Use consistent service names** across all operations from the same service.
+4. **Use descriptive service names** that identify the data source.
+5. **Document the field names** used by your service to help users understand what fields are available for searching.
+
+## Implementation Details
+
+Service identification is implemented through:
+
+1. The `getServiceName` helper method in `BrainyData` class
+2. The `trackFieldNames` method in `StorageAdapter` interface
+3. The `incrementStatistic` and `decrementStatistic` methods in `StorageAdapter` interface
+
+These methods ensure that statistics and field names are properly tracked by service.
--- a/src/brainyData.ts
+++ b/src/brainyData.ts
@ -55,6 +55,13 @@ export interface BrainyDataConfig {
   */
  hnsw?: Partial<HNSWOptimizedConfig>

+  /**
+   * Default service name to use for all operations
+   * When specified, this service name will be used for all operations
+   * that don't explicitly provide a service name
+   */
+  defaultService?: string
+
  /**
   * Distance function to use for similarity calculations
   */
@ -344,6 +351,7 @@ export class BrainyData<T = any> implements BrainyDataInterface<T> {
  private useOptimizedIndex: boolean = false
  private _dimensions: number
  private loggingConfig: BrainyDataConfig['logging'] = { verbose: true }
+  private defaultService: string = 'default'

  // Timeout and retry configuration
  private timeoutConfig: BrainyDataConfig['timeouts'] = {}
@ -455,6 +463,11 @@ export class BrainyData<T = any> implements BrainyDataInterface<T> {
      throw new Error('Database cannot be both read-only and write-only')
    }

+    // Set default service name if provided
+    if (config.defaultService) {
+      this.defaultService = config.defaultService
+    }
+
    // Store storage configuration for later use in init()
    this.storageConfig = config.storage || {}

@ -873,7 +886,7 @@ export class BrainyData<T = any> implements BrainyDataInterface<T> {
  }

  /**
-   * Get the service name from options or fallback to current augmentation
+   * Get the service name from options or fallback to default service
   * This provides a consistent way to handle service names across all methods
   * @param options Options object that may contain a service property
   * @returns The service name to use for operations
@ -882,7 +895,9 @@ export class BrainyData<T = any> implements BrainyDataInterface<T> {
    if (options?.service) {
      return options.service
    }
-    return this.getCurrentAugmentation()
+    // Use the default service name specified during initialization
+    // This simplifies service identification by allowing it to be specified once
+    return this.defaultService
  }

  /**
@ -2377,7 +2392,7 @@ export class BrainyData<T = any> implements BrainyDataInterface<T> {
        }

        // Get the service that's updating the metadata
-        const service = options.service || this.getCurrentAugmentation()
+        const service = this.getServiceName(options)
        const graphNoun = metadata as unknown as GraphNoun

        // Preserve existing createdBy and createdAt if they exist
@ -2424,7 +2439,7 @@ export class BrainyData<T = any> implements BrainyDataInterface<T> {
      await this.storage!.saveMetadata(id, metadata)

      // Track metadata statistics
-      const service = options.service || this.getCurrentAugmentation()
+      const service = this.getServiceName(options)
      await this.storage!.incrementStatistic('metadata', service)

      return true
@ -2534,7 +2549,7 @@ export class BrainyData<T = any> implements BrainyDataInterface<T> {
          const placeholderVector = new Array(this._dimensions).fill(0)

          // Add metadata if provided
-          const service = options.service || this.getCurrentAugmentation()
+          const service = this.getServiceName(options)
          const now = new Date()
          const timestamp = {
            seconds: Math.floor(now.getTime() / 1000),
@ -2576,7 +2591,7 @@ export class BrainyData<T = any> implements BrainyDataInterface<T> {
          const placeholderVector = new Array(this._dimensions).fill(0)

          // Add metadata if provided
-          const service = options.service || this.getCurrentAugmentation()
+          const service = this.getServiceName(options)
          const now = new Date()
          const timestamp = {
            seconds: Math.floor(now.getTime() / 1000),
@ -2685,7 +2700,7 @@ export class BrainyData<T = any> implements BrainyDataInterface<T> {
      // Note: We're no longer validating against VerbType enum to allow custom relationship types

      // Get service name from options or current augmentation
-      const service = options.service || this.getCurrentAugmentation()
+      const service = this.getServiceName(options)

      // Create timestamp for creation/update time
      const now = new Date()
@ -3271,6 +3286,78 @@ export class BrainyData<T = any> implements BrainyDataInterface<T> {
    }
  }

+  /**
+   * Calculate similarity between two vectors or between two pieces of text/data
+   * This method allows clients to directly calculate similarity scores between items
+   * without needing to add them to the database
+   *
+   * @param a First vector or text/data to compare
+   * @param b Second vector or text/data to compare
+   * @param options Additional options
+   * @returns A promise that resolves to the similarity score (higher means more similar)
+   */
+  public async calculateSimilarity(
+    a: Vector | string | string[],
+    b: Vector | string | string[],
+    options: {
+      forceEmbed?: boolean // Force using the embedding function even if input is a vector
+      distanceFunction?: DistanceFunction // Optional custom distance function
+    } = {}
+  ): Promise<number> {
+    await this.ensureInitialized()
+
+    try {
+      // Convert inputs to vectors if needed
+      let vectorA: Vector
+      let vectorB: Vector
+
+      // Process first input
+      if (
+        Array.isArray(a) &&
+        a.every((item) => typeof item === 'number') &&
+        !options.forceEmbed
+      ) {
+        // Input is already a vector
+        vectorA = a
+      } else {
+        // Input needs to be vectorized
+        try {
+          vectorA = await this.embeddingFunction(a)
+        } catch (embedError) {
+          throw new Error(`Failed to vectorize first input: ${embedError}`)
+        }
+      }
+
+      // Process second input
+      if (
+        Array.isArray(b) &&
+        b.every((item) => typeof item === 'number') &&
+        !options.forceEmbed
+      ) {
+        // Input is already a vector
+        vectorB = b
+      } else {
+        // Input needs to be vectorized
+        try {
+          vectorB = await this.embeddingFunction(b)
+        } catch (embedError) {
+          throw new Error(`Failed to vectorize second input: ${embedError}`)
+        }
+      }
+
+      // Calculate distance using the specified or default distance function
+      const distanceFunction = options.distanceFunction || this.distanceFunction
+      const distance = distanceFunction(vectorA, vectorB)
+
+      // Convert distance to similarity score (1 - distance for cosine)
+      // Higher value means more similar
+      return 1 - distance
+    } catch (error) {
+      console.error('Failed to calculate similarity:', error)
+      throw new Error(`Failed to calculate similarity: ${error}`)
+    }
+  }
+
  /**
   * Search for verbs by type and/or vector similarity
   * @param queryVectorOrData Query vector or data to search for