Session ID

A transaction refers to an individual API call and is identified by a unique* transaction_id.

A session is a collection of transactions that are grouped together based on certain criteria, and each share the same session_id. For instance, a session might consist of a sequence of user interactions with an end application, leading to several consecutive LLM-API calls.

gecholog doesn't dictate the definition of a session in your specific use cases but provides the functionality to track both session_id and transaction_id.

Session ID format

The session_id ends with 0 in this format:

TST00001_1699542383504801033_23498_0

Transaction ID format

The transaction_id ends with 0...n in this format:

TST00001_1699542383504801033_23498_0
TST00001_1699542383504801033_23498_1
TST00001_1699542383504801033_23498_2
...

The log

Both session_id and transaction_id can be found in the log from gecholog

{
    "session_id": "TST00001_1699542383504801033_234_0",
    "transaction_id": "TST00001_1699542383504801033_234_1"
}

Read more about the log structure in Log Reference.

Receiving session/transaction ID

For every api call to the gecholog router, gecholog will response with a Session-Id header:

Session-Id: TST00001_1699884006500487748_1_0

When it's the first call in a session, this value represents the session_id
If it's not the first call in the session, the session_id header will instead contain the transaction_id value

Creating a session

By default, gecholog assumes that each transaction is its own session. In order to add a transaction to a session, the ingress request to gecholog needs to contain a valid Session-Id request header populated with the previous transaction_id of the session.

Example cUrl request with a Session-Id request header.

Windows Command Line

setx OPENAI_API_KEY "your_api_key"              
setx DEPLOYMENT "your_azure_deployment"

MacOS/Linux

export OPENAI_API_KEY=your_api_key      
export DEPLOYMENT=your_azure_deployment

Windows Command Line

curl -i -X POST ^
     -H "api-key: %OPENAI_API_KEY%" ^
     -H "Session-Id: TST00001_1699884006500487748_1_0" ^
     -H "Content-Type: application/json" ^
     -d "{\"messages\": [{\"role\": \"system\",\"content\": \"Assistant is a large language model trained by OpenAI.\"},{\"role\": \"user\",\"content\": \"Who are the founders of Microsoft?\"}],\"max_tokens\": 15}" ^
     http://localhost:5380/service/standard/openai/deployments/%DEPLOYMENT%/chat/completions?api-version=2023-12-01-preview

MacOS/Linux

curl -i -X POST -H "api-key: $OPENAI_API_KEY" -H "Session-Id: TST00001_1699884006500487748_1_0" -H "Content-Type: application/json" -d '{
    "messages": [
      {
        "role": "system",
        "content": "Assistant is a large language model trained by OpenAI."
      },
      {
        "role": "user",
        "content": "Who are the founders of Microsoft?"
      }
    ],
    "max_tokens": 15
  }' "http://localhost:5380/service/standard/openai/deployments/$DEPLOYMENT/chat/completions?api-version=2023-12-01-preview"

And we receive the transaction_id as a response Session-Id header.

Session-Id: TST00001_1699884006500487748_1_1

NOTE: The name of the Session-Id header can be changed in the /app/conf/gl_config.json file.

* `transaction_id` unique?

Given that gecholog operates in a stateless manner, it doesn't verify whether the session_id header actually corresponds to the previous transaction_id of the session; it assumes this to be true. Therefore, if the session_id header received at ingress is identical for two separate transactions, the transaction_id will be the same for both.