Processor Rules

What is the first rule of processors? That gecholog determines the rules for the processors.

Process Ordering

What processors are executed when

This is the gecholog flow.

Processors are executed in this order

1. Request processors sync
2. Response processors sync
3. Request processors async
4. Response processors async

Sequential and parallel

In each phase, processors can be executed in parallel or in sequence.

Each setting for a processor in the /app/conf/gl_config.json file is called its context. Read more about the settings in Section Processor configs.

---

NOTE: In graphics above: processor 1 reused multiple times. Each is a different context.

---

Processor orderings in web interface

The web interface can be used to determine the execution order of processors. Example for request processors where processors_1a, processors_1b and processors_1c are executed in parallel before processors_2 etc.

Asynchronous processors in web interface

Asynchronous processors are highlighten in the web interface. They will be executed after the payload is sent back to the requestor. In this example processors_1b and processors_3b are asynchronous.

Add or modify

The modifier=true/false flag determines if the processor can add OR modify fields. A processor cannot do both in the same context.

modifier=true marks a processor that can

• Completely or partially change one or many fields
• Cannot add fields to the request/response object

modifier=false marks a processor that can

• Create and add one or more fields in the request/response object
• Cannot modify existing fields

Input to processor

In each context, the input_fields_include and input_fields_exclude settings determine what fields in the request/response object gecholog sends to the processor.

Output from processor

In each context, the output_fields_write setting determine what fields in the request/response object gecholog will allow the processor to modify/add. If the processor sends back other fields, they will be ignored. If a processor has modified an existing field but modifier=false the changes will be ignored. If the processor has created new fields but modifier=true the new fields will be ignored.

Request or response

Every processor works EITHER on the request OR the response object, not on both in the same context.

Request sync recommendations

A request sync modifier (modifier=true and async=false) will technically be allowed to modify all request object fields. However, it is recommended to only modify the fields with outbound prefix. Why?

• It keeps better audit trail "this was the ingress, this is what we sent outbound after processing"
• Modifying fields with ingress prefix doesn't impact the actual request to the target. It more serves the purpose to redact the content, which is better done async.

Response sync recommendations

A response sync modifier (modifier=true and async=false) will technically be allowed to modify all response object fields. However, it is recommended to only modify the fields with egress prefix. Why?

• It keeps better audit trail "this was the inbound, this is what we sent egress after processing"
• Modifying fields with inbound prefix doesn't impact the actual response to the requestor. It more serves the purpose to redact the content, which is better done async.

Required

The required=true option is generally not advised for most use cases. If a processor fails to complete and required=true is set, the process will be halted at that point. It's better to avoid this setting unless absolutely necessary. An exception might be in scenarios where an asynchronous processor must redact PII data; if it doesn't succeed, let's assume you for data privacy reasons are not allow to store the log at all. In such cases, required=true becomes essential as a fail-safe for redaction errors. Note that if a processor concludes with completed=false when required=true is set, there will be no log generated.

Loggers

Loggers are a special kind of processors that are not ruled by gecholog and are not configured in the /app/conf/gl_config.json file. Loggers only subscribe to the topic_logger_exact topic of the service bus and takes it's action from there. nats2log is a logger. Read more about loggres in Section Standard Library Processors.

Bend the rules

Although gecholog strictly defines the permissible actions for a processor within a given context, there's still room for creativity. Processors can be repurposed across different contexts, meaning a processor limited to modifying or adding specific field types in one context can be adapted to handle different fields in different way in another. This flexibility allows for the creation of versatile, multipurpose processors. An example from the Standard Library is tokencounter, which functions both as a synchronous request processor and an asynchronous response processor.

Example

tokencounter in context request/sync[0]

{
    "name": "token_counter",
    "modifier": false,
    "required": false,
    "async": false,
    "input_fields_include": [ "ingress_payload", "gl_path" ],
    "input_fields_exclude": [ ],
    "output_fields_write": [ "control" ],
    "service_bus_topic": "coburn.gl.tokencounter",
    "timeout": 50
}

tokencounter in context response/async[0]

{
    "name": "token_counter",
    "modifier": false,
    "required": false,
    "async": true,
    "input_fields_include": [ "inbound_payload", "gl_path" ],
    "input_fields_exclude": [ ],
    "output_fields_write": [ "token_count" ],
    "service_bus_topic": "coburn.gl.tokencounter",
    "timeout": 50
}