Standard Library Processors

The gecholog container contains the standard library processors. Standard Library means they are maintained and supported by the gecholog.ai team. The standard library processors are:

• nats2log subscribes to logger topic and pushes logs using either file_writer rest_api_writer elastic_writer or azure_log_analytics_writer
• tokencounter to add token_count fields and throttle traffic

Each processor has their own configuration file and can easily be disabled.

NOTE: nats2file is just a second instance of nats2log with default mode file_writer

nats2log

Config file app/conf/nats2log_config.json

{
  "version": "0.92.1",
  "log_level": "INFO",
  "mode": "azure_log_analytics_writer",
  "retries": 3,
  "retry_delay_milliseconds": 250,
  "service_bus": {
    "hostname": "localhost:4222",
    "topic": "coburn.gl.nats2log",
    "topic_exact_logger": "coburn.gl${NATS2LOG_LOGGER_SUBTOPIC}",
    "token": "${NATS_TOKEN}"
  },
  "tls": {
    "insecure": false,
    "system_cert_pool": true,
    "cert_files": [ ]
  },
  "file_writer": {
    "filename": "/app/log/disabled.jsonl",
    "write_mode": "append"
  },
  "elastic_writer": {
    "url": "https://localhost",
    "port": 9200,
    "index": "coburn1",
    "username": "elastic",
    "password": "${ELASTIC_PASSWORD}"
  },
  "azure_log_analytics_writer": {
    "workspace_id": "${AZURE_LOG_ANALYTICS_WORKSPACE_ID}",
    "shared_key": "${AZURE_LOG_ANALYTICS_SHARED_KEY}",
    "log_type": "gecholog"
  },
  "rest_api_writer": {
    "url": "https://localhost",
    "port": 443,
    "endpoint": "/log",
    "headers": 
      {
        "headers1": [ "value1"]
      }
  }
}

environment variables

nats2log supports environment variables described in section Environment Variables.

service_bus

nats2log subscribes to the service bus topic where gecholog posts the final logs. nats2log sends them to 3rd party log storage using different writers. The processor nats2log executes after the post-processing chain. It does not alter any message in gecholog in any way and only subscribes to the topic_exact_logger topic. topic_exact_logger is set using the ${NATS2LOG_LOGGER_SUBTOPIC} environment variable by default. Leave ${NATS2LOG_LOGGER_SUBTOPIC} empty and nats2log will subscribe to a mute coburn.gl topic, hence being disabled. Set ${NATS2LOG_LOGGER_SUBTOPIC}=.logger to enable. You can also remove the dependecy on the environment variable and enable nats2log by setting topic_exact_logger=coburn.gl.logger.

tls

azure_log_analytics_writer

azure_log_analytics_writer posts the logs to the data collections /api/logs api. Since there are many moving parts when connecting to a new external service, it's recommended to temporarily set log_level=DEBUG to capture more details related to issues with the POST requests in the container system log. Make sure tls options are configured with at least system_cert_pool=true.

elastic_writer

elastic_writer posts the logs to the Elastic /_doc api. Since there are many moving parts when connecting to a new external service, it's recommended to temporarily set log_level=DEBUG to capture more details related to issues with the POST requests in the container system log. For https make sure tls options are configured to support connecting to the url of Elastic.

file_writer

append adds lines to the existing file or creates a new one if the file doesn't already exist. overwrite clears the file and then appends to it. new creates new filenames _1 _2 ... _n if the files already exist, then appends to the new file.

If you want to mount a local host path to be able to ingest/analyze the logs, you can start the gecholog container by mounting a local volume like this

docker run -d --name gecholog -p 5380:5380 -p 8080:8080 -e -e GUI_SECRET=changeme AISERVICE_API_BASE=https://your.openai.azure.com/ -e NATS2FILE_LOGGER_SUBTOPIC=.logger -v your/path:/app/log gecholog/gecholog:latest

az container create --resource-group <RESOURCE_GROUP> --name gecholog --image gecholog/gecholog:latest --azure-file-volume-account-name <STORAGE_ACCOUNT_NAME> --azure-file-volume-account-key <STORAGE_ACCOUNT_KEY> --azure-file-volume-share-name <FILE_SHARE_NAME> --azure-file-volume-mount-path /app/log --environment-variables AISERVICE_API_BASE=https://your.openai.azure.com/ NATS2FILE_LOGGER_SUBTOPIC=.logger --secure-environment-variables GUI_SECRET=changeme --dns-name-label gecholog-changeme --ports 5380 8080

Or add these lines to your docker-compose.yml file

version: '3'
services:
  gecholog:
    image: gecholog/gecholog:latest
    environment:
      - NATS2FILE_LOGGER_SUBTOPIC=.logger
    volumes:
      - /path/on/your/host:/app/log

rest_api_writer

rest_api_writer is suitable for writing to a REST api that only requires static headers. For https make sure tls options are configured to support connecting to the url.

Configuration validation

nats2log employs strict configuration validation of the configuration structure and configuration fields. To support users to process of creating valid configuration files, nats2log supports the --validate flag. For more information about config file validation visit section Field Validation.

nats2file

nats2file is just one more instance of nats2log with default mode file_writer with configuration file /app/conf/nats2file_config.json. Read more about the configuration options in Section file_writer. For topic_exact_logger nats2file uses the environment variable ${NATS2FILE_LOGGER_SUBTOPIC} by default. Use nats2log to validate a new configuration file also for nats2file.

tokencounter

Config file tokencounter_config.json

{
    "version": "0.92.1",
    "log_level": "INFO",
    "service_bus": {
        "hostname": "localhost:4222",
        "topic": "coburn.gl.tokencounter",
        "token": "${NATS_TOKEN}"
    },
    "cap_period_seconds": 120,
    "token_caps": [
        {
            "router": "/service/capped/",
            "fields": [
                {
                    "field": "prompt_tokens",
                    "value": 500
                },
                {
                    "field": "completion_tokens",
                    "value": 500
                },
                {
                    "field": "total_tokens",
                    "value": 100
                }
            ]
        },
        {
            "router": "/service/standard/",
            "fields": [
                {
                    "field": "prompt_tokens",
                    "value": 0
                },
                {
                    "field": "completion_tokens",
                    "value": 0
                },
                {
                    "field": "total_tokens",
                    "value": 0
                }
            ]
        }
    ],
    "usage_fields": [
        {
            "router": "default",
            "patterns": [
                {
                    "field": "prompt_tokens",
                    "pattern": "inbound_payload.usage.prompt_tokens"
                },
                {
                    "field": "completion_tokens",
                    "pattern": "inbound_payload.usage.completion_tokens"
                },
                {
                    "field": "total_tokens",
                    "pattern": "inbound_payload.usage.total_tokens"
                }
            ]
        }
    ]
}

service_bus

token_caps

usage_fields

json-search path uses gjson syntax gjson on github.

Configuration validation

tokencounter uses strict configuration validation of the configuration structure and configuration fields. To support users to process of creating valid configuration files, tokencounter supports the --validate flag. For more information about config file validation visit section Field Validation.

Usage

tokencounter can be used as

• response processor only OR
• request and response processor

As a response processor tokencounter expects inbound_payload and gl_path fields. It creates a new to token_count field in the response object. It keeps keeps track of the token consumption for the fields prompt, completion and total for each path in the token_caps map.

As a request processor tokencounter uses the tracked consumption from the response traffic to determine if the threshold for token_caps has been surpassed or not. It will block traffic to the target if token consumption > threshold when a new request arrives. It will send back the response

{
  "Error": "Consumption Cap Exceeded. Try again later",
  "Path":"/service/capped/"
}

It will reset all counters every cap_period_seconds seconds.