# Metrics

Secret Agent publishes metrics to a Prometheus endpoint and to the log ticker.

## Prometheus configuration

To expose a Prometheus metrics endpoint, configure the HTTP or HTTPS service in the Secret Agent configuration file:

```yaml
service:

  http:

    endpoint: 0.0.0.0:8080

    metrics:

      prometheus: {}
```

With the default base path, the Prometheus endpoint is at `/manage/rest/v1/prometheus` (full URL for HTTP: `http://localhost:8080/manage/rest/v1/prometheus`). You can set a custom base path with `service.http.metrics.prometheus.url-base-path` or `service.https.metrics.prometheus.url-base-path`. See [Configure HTTP and HTTPS](https://aerospike.com/docs/database/tools/secret-agent/listener/http) for validation rules and examples.

Secret Agent exposes metrics for:

-   Listeners (TCP, UDS)
-   Secret managers (AWS, GCP)

All metrics include a `module` label whose value identifies the module emitting the metric.

### Listener metrics

| Metric name | Description | Modules | Type |
| --- | --- | --- | --- |
| `aerospike_sa_connections_active` | Number of active connections | TCP, UDS | Gauge |
| `aerospike_sa_connections_open` | Number of opened connections | TCP, UDS | Counter |
| `aerospike_sa_connections_closed` | Number of closed connections | TCP, UDS | Counter |
| `aerospike_sa_connections_err` | Number of connection errors | TCP, UDS | Counter |
| `aerospike_sa_read_err` | Number of read errors on connections | TCP, UDS | Counter |
| `aerospike_sa_write_err` | Number of write errors on connections | TCP, UDS | Counter |
| `aerospike_sa_tls_err` | Number of TLS handshake errors | TCP, UDS | Counter |
| `aerospike_sa_parse_err` | Number of request parsing errors | TCP, UDS | Counter |

### Secret manager metrics

| Metric name | Description | Modules | Type |
| --- | --- | --- | --- |
| `aerospike_sa_success` | Number of successful fetch requests | AWS, GCP | Counter |
| `aerospike_sa_fetch_latency_p50` | p50 latency for fetch requests | AWS, GCP | Gauge |
| `aerospike_sa_fetch_latency_p95` | p95 latency for fetch requests | AWS, GCP | Gauge |
| `aerospike_sa_resource_err` | Number of errors from invalid resource names in requests | AWS, GCP | Counter |
| `aerospike_sa_fetch_err` | Number of errors fetching secrets from the external secret manager | AWS, GCP | Counter |
| `aerospike_sa_key_not_found_err` | Number of errors where the requested key is not found in the secret | AWS | Counter |
| `aerospike_sa_session_err` | Number of errors creating a client session to the external secret manager | AWS, GCP | Counter |
| `aerospike_sa_role_err` | Number of errors assuming a role (AWS) or impersonating (GCP) | AWS, GCP | Counter |

## Custom labels

You can add custom labels to Prometheus metrics. Define labels as key-value pairs in the `metrics` context:

```yaml
service:

  http:

    endpoint: 0.0.0.0:8080

    metrics:

      prometheus:

        labels:

          label1: val1

          label2: val2
```

## Log ticker

In addition to Prometheus metrics, Secret Agent prints metrics to the log every 10 seconds.

The log ticker format is:

`<module name>: <metric1> <val1>, <metric2> <val2>, ...`

Example log ticker output:

```plaintext
[INFO] 2023/09/09 11:22:34 metrics.go:65: tcp: connections_active 1, connections_opened 2, connections_closed 1, connections_err 0, read_err 0, write_err 0, tls_err 0, parse_err 0

[INFO] 2023/09/09 11:22:34 metrics.go:65: uds: connections_active 0, connections_opened 0, connections_closed 0, connections_err 0, read_err 0, write_err 0, tls_err 0, parse_err 0

[INFO] 2023/09/09 11:22:34 metrics.go:65: aws: success 2, fetch_latency_p50 38, fetch_latency_p95 108, resource_err 0, fetch_err 0, key_not_found_err 0, session_err 0, role_err 0
```