# Temporal SDK metrics reference

> Temporal SDKs emit metrics covering Client usage and Worker Processes. Metrics can be tuned to improve Worker performance and are prefixed with temporal_ before export.

> **ℹ️ Info:**
> SDK metrics
>
> The information on this page is relevant to [Temporal SDKs](/encyclopedia/temporal-sdks).
>
> See [Cloud metrics](/cloud/metrics/) for metrics emitted by [Temporal Cloud](/cloud/overview).
>
> See [Cluster metrics](/references/cluster-metrics) for metrics emitted by the [OSS Cluster](/temporal-service).
>
> Some SDKs may emit metrics beyond what is listed in this SDK Metrics reference.
> Only metrics included in this Metrics reference have guaranteed, defined behavior.
> Other metrics are considered deprecated, inconsistent or experimental.
>

The Temporal SDKs emit a set of metrics from Temporal Client usage and Worker Processes.

- [How to emit metrics using the Go SDK](/develop/go/platform/observability#metrics)
- [How to emit metrics using the Java SDK](/develop/java/platform/observability#metrics)
- [How to emit metrics using the Python SDK](/develop/python/platform/observability#metrics)
- [How to emit metrics using the TypeScript SDK](/develop/typescript/platform/observability#metrics)
- [How to emit metrics using the .NET SDK](/develop/dotnet/platform/observability#metrics)
- [How to emit metrics using the Ruby SDK](/develop/ruby/platform/observability#metrics)
- [How to tune Worker performance based on metrics](/develop/worker-performance)

All metrics are prefixed with `temporal_` before being exported to their configured destination.
(The prefix has been removed in parts of this reference.)
Currently, some metrics are specific to certain SDKs.

TypeScript, Python, .NET, and Ruby SDK metrics are defined in the Core SDK.

PHP and Go metrics are defined in the Go SDK.

Java metrics are defined in the Java SDK.
Metrics are defined in the following locations.

- [Core SDK Worker metrics](https://github.com/temporalio/sdk-rust/blob/main/crates/sdk-core/src/telemetry/metrics.rs)
- [Core SDK Client metrics](https://github.com/temporalio/sdk-rust/blob/main/crates/client/src/metrics.rs)
- [Java SDK Worker metrics](https://github.com/temporalio/sdk-java/blob/master/temporal-sdk/src/main/java/io/temporal/worker/MetricsType.java)
- [Java SDK Client metrics](https://github.com/temporalio/sdk-java/blob/master/temporal-serviceclient/src/main/java/io/temporal/serviceclient/MetricsType.java)
- [Go SDK Worker and Client metrics](https://github.com/temporalio/sdk-go/blob/c32b04729cc7691f80c16f80eed7f323ee5ce24f/internal/common/metrics/constants.go)

> **📝 Note:**
> Metric units across SDKs
>
> The unit of measurement for metrics can vary based on which SDK they are being reported from:
>
> **Core-based SDKs:** Metrics of the type Histogram are measured in _milliseconds_ by default.
> This can be customized to use seconds for SDKs using [Core SDK](/glossary#core-sdk).
> The Core SDK is a shared common core library used by several Temporal SDKs, including TypeScript, Python, and .NET.
>
> **Java and Go SDKs:** Metrics of the type Histogram are measured in _seconds_.
>

Each metric may have some combination of the following keys attached to them:

- `task-queue`: Task Queue that the Worker Entity is polling
- `namespace`: Namespace the Worker is bound to
- `poller_type`: One of the following:
  - `workflow_task`
  - `activity_task`
  - `nexus_task` (Go and Java only)
  - `sticky_workflow_task`
- `worker_type`: One of the following:
  - `ActivityWorker`
  - `WorkflowWorker`
  - `LocalActivityWorker` (Go and Java only)
  - `NexusWorker` (Go and Java only)
- `activity_type`: The name of the Activity Function the metric is associated with
- `workflow_type`: The name of the Workflow Function the metric is associated with
- `operation`: RPC method name; available for metrics related to Temporal Client gRPC requests

Some keys may not be available in every SDK, and Histogram metrics may have different buckets in each SDK.

| Metric name                                                                                      | Emitted by     | Metric type | Availability   |
| ------------------------------------------------------------------------------------------------ | -------------- | ----------- | -------------- |
| [temporal_activity_execution_cancelled](#activity_execution_cancelled)                           | Worker         | Counter     | Java           |
| [temporal_activity_execution_failed](#activity_execution_failed)                                 | Worker         | Counter     | Core, Go, Java |
| [temporal_activity_execution_latency](#activity_execution_latency)                               | Worker         | Histogram   | Core, Go, Java |
| [temporal_activity_poll_no_task](#activity_poll_no_task)                                         | Worker         | Counter     | Core, Go, Java |
| [temporal_activity_schedule_to_start_latency](#activity_schedule_to_start_latency)               | Worker         | Histogram   | Core, Go, Java |
| [temporal_activity_succeed_endtoend_latency](#activity_succeed_endtoend_latency)                 | Worker         | Histogram   | Core, Go, Java |
| [temporal_activity_task_error](#activity_task_error)                                             | Worker         | Counter     | Go             |
| [temporal_corrupted_signals](#corrupted_signals)                                                 | Worker         | Counter     | Go, Java       |
| [temporal_local_activity_execution_cancelled](#local_activity_execution_cancelled)               | Worker         | Counter     | Core, Go, Java |
| [temporal_local_activity_execution_failed](#local_activity_execution_failed)                     | Worker         | Counter     | Core, Go, Java |
| [temporal_local_activity_execution_latency](#local_activity_execution_latency)                   | Worker         | Histogram   | Core, Go, Java |
| [temporal_local_activity_succeeded_endtoend_latency](#local_activity_succeeded_endtoend_latency) | Worker         | Histogram   | Core, Go, Java |
| [temporal_local_activity_total](#local_activity_total)                                           | Worker         | Counter     | Core, Go, Java |
| [temporal_long_request](#long_request)                                                           | Service Client | Counter     | Core, Go, Java |
| [temporal_long_request_failure](#long_request_failure)                                           | Service Client | Counter     | Core, Go, Java |
| [temporal_long_request_latency](#long_request_latency)                                           | Service Client | Histogram   | Core, Go, Java |
| [temporal_nexus_poll_no_task](#nexus_poll_no_task)                                               | Worker         | Counter     | Core, Go, Java |
| [temporal_nexus_task_schedule_to_start_latency](#nexus_task_schedule_to_start_latency)           | Worker         | Histogram   | Core, Go, Java |
| [temporal_nexus_task_execution_failed](#nexus_task_execution_failed)                             | Worker         | Counter     | Core, Go, Java |
| [temporal_nexus_task_execution_latency](#nexus_task_execution_latency)                           | Worker         | Histogram   | Core, Go, Java |
| [temporal_nexus_task_endtoend_latency](#nexus_task_endtoend_latency)                             | Worker         | Histogram   | Core, Go, Java |
| [temporal_num_pollers](#num_pollers)                                                             | Worker         | Gauge       | Core, Go       |
| [temporal_poller_start](#poller_start)                                                           | Worker         | Counter     | Go, Java       |
| [temporal_request](#request)                                                                     | Service Client | Counter     | Core, Go, Java |
| [temporal_request_failure](#request_failure)                                                     | Service Client | Counter     | Core, Go, Java |
| [temporal_request_latency](#request_latency)                                                     | Service Client | Histogram   | Core, Go, Java |
| [temporal_resource_slots_mem_usage](#resource_slots_cpu_usage)                                   | Worker         | Gauge       | Core, Java     |
| [temporal_resource_slots_cpu_usage](#resource_slots_mem_usage)                                   | Worker         | Gauge       | Core, Java     |
| [temporal_sticky_cache_hit](#sticky_cache_hit)                                                   | Worker         | Counter     | Core, Go, Java |
| [temporal_sticky_cache_miss](#sticky_cache_miss)                                                 | Worker         | Counter     | Core, Go, Java |
| [temporal_sticky_cache_size](#sticky_cache_size)                                                 | Worker         | Gauge       | Core, Go, Java |
| [temporal_sticky_cache_total_forced_eviction](#sticky_cache_total_forced_eviction)               | Worker         | Counter     | Go, Java       |
| [temporal_unregistered_activity_invocation](#unregistered_activity_invocation)                   | Worker         | Counter     | Go             |
| [temporal_worker_start](#worker_start)                                                           | Worker         | Counter     | Core, Go, Java |
| [temporal_worker_task_slots_available](#worker_task_slots_available)                             | Worker         | Gauge       | Core, Go, Java |
| [temporal_worker_task_slots_used](#worker_task_slots_used)                                       | Worker         | Gauge       | Core, Go, Java |
| [temporal_workflow_active_thread_count](#workflow_active_thread_count)                           | Worker         | Gauge       | Java           |
| [temporal_workflow_cancelled](#workflow_cancelled)                                               | Worker         | Counter     | Core, Go, Java |
| [temporal_workflow_completed](#workflow_completed)                                               | Worker         | Counter     | Core, Go, Java |
| [temporal_workflow_continue_as_new](#workflow_continue_as_new)                                   | Worker         | Counter     | Core, Go, Java |
| [temporal_workflow_endtoend_latency](#workflow_endtoend_latency)                                 | Worker         | Histogram   | Core, Go, Java |
| [temporal_workflow_failed](#workflow_failed)                                                     | Worker         | Counter     | Core, Go, Java |
| [temporal_workflow_task_execution_failed](#workflow_task_execution_failed)                       | Worker         | Counter     | Core, Go, Java |
| [temporal_workflow_task_execution_latency](#workflow_task_execution_latency)                     | Worker         | Histogram   | Core, Go, Java |
| [temporal_workflow_task_queue_poll_empty](#workflow_task_queue_poll_empty)                       | Worker         | Counter     | Core, Go, Java |
| [temporal_workflow_task_queue_poll_succeed](#workflow_task_queue_poll_succeed)                   | Worker         | Counter     | Core, Go, Java |
| [temporal_workflow_task_replay_latency](#workflow_task_replay_latency)                           | Worker         | Histogram   | Core, Go, Java |
| [temporal_workflow_task_schedule_to_start_latency](#workflow_task_schedule_to_start_latency)     | Worker         | Histogram   | Core, Go, Java |

### activity_execution_cancelled

An Activity Execution was canceled.

- Type: Counter
- Available in: Java
- Tags: `activity_type`, `namespace`, `task_queue`

### activity_execution_failed

An Activity Execution failed.
This does not include local Activity Failures in the Go and Java SDKs (see [local_activity_execution_failed](#local_activity_execution_failed)).

- Type: Counter
- Available in: Core, Go, Java
- Tags: `activity_type`, `namespace`, `task_queue`

### activity_execution_latency

Time to complete an Activity Execution, from the time the Activity Task is generated to the time the language SDK responded with a completion (failure or success).

- Type: Histogram
- Available in: Core, Go, Java
- Tags: `activity_type`, `namespace`, `task_queue`

### activity_poll_no_task

An Activity Worker poll for an Activity Task timed out, and no Activity Task is available to pick from the Task Queue.

- Type: Counter
- Available in: Core, Go, Java
- Tags: `namespace`, `task_queue`

### activity_schedule_to_start_latency

The Schedule-To-Start time of an Activity Task in seconds.
A [Schedule-To-Start Timeout](/encyclopedia/detecting-activity-failures#schedule-to-start-timeout) can be set when an Activity Execution is spawned.
This metric is useful for ensuring Activity Tasks are being processed from the queue in a timely manner. Some SDKs may include
the `activity_type` label, but the metric should not vary by type, as it does not influence the rate at which tasks are pulled
from the queue.

- Type: Histogram
- Available in: Core, Go, Java
- Tags: `namespace`, `task_queue`

### activity_succeed_endtoend_latency

Total latency of successfully finished Activity Executions from the time they are scheduled to the time they are completed.
This metric is not recorded for async Activity completion.

- Type: Histogram
- Available in: Core, Go, Java
- Tags: `activity_type`, `namespace`, `task_queue`

### activity_task_error

An internal error or panic occurred during Activity Task handling or execution.

- Type: Counter
- Available in: Go,
- Tags: `activity_type`, `namespace`, `task_queue`, `workflow_type`

### corrupted_signals

Number of Signals whose payload could not be deserialized.

- Type: Counter
- Available in: Go, Java
- Tags: `namespace`, `task_queue`, `workflow_type`

### local_activity_execution_cancelled

A Local Activity Execution was canceled.

- Type: Counter
- Available in: Core, Go, Java
- Tags: `activity_type`, `namespace`, `task_queue`

### local_activity_execution_failed

A Local Activity Execution failed.

- Type: Counter
- Available in: Core, Go, Java
- Tags: `activity_type`, `namespace`, `task_queue`

### local_activity_execution_latency

Time to complete a Local Activity Execution, from the time the first Activity Task is generated to the time the SDK responds that the execution is complete.

- Type: Histogram
- Available in: Core, Go, Java
- Tags: `activity_type`, `namespace`, `task_queue`

### local_activity_succeeded_endtoend_latency

Total latency of successfully finished Local Activity Executions (from schedule to completion).

- Type: Histogram
- Available in: Core, Go, Java
- Tags: `activity_type`, `namespace`, `task_queue`

### local_activity_total

Total number of [Local Activity Executions](/local-activity).

- Type: Counter
- Available in: Core, Go, Java
- Tags: `activity_type`, `namespace`, `task_queue`

### long_request

Temporal Client made an RPC long poll request.

- Type: Counter
- Available in: Core, Go, Java
- Tags: `namespace`, `operation`

### long_request_failure

Temporal Client made an RPC long poll request that failed.
This number is included into the total `long_request` counter for long poll RPC requests.

- Type: Counter
- Available in: Core, Go, Java
- Tags: `namespace`, `operation`

### long_request_latency

Latency of a Temporal Client gRPC long poll request.

- Type: Histogram
- Available in: Core, Go, Java
- Tags: `namespace`, `operation`

### nexus_poll_no_task

A Nexus Worker poll for a Nexus Task timed out, and no Nexus Task is available to pick from the Task Queue.

- Type: Counter
- Available in: Go, Java
- Tags: `namespace`, `task_queue`

### nexus_task_schedule_to_start_latency

The Schedule-To-Start time of a Nexus Task in seconds. The schedule time is taken from when the corresponding request
hit the Frontend service to when the SDK started processing the task.

This time is limited by the `Request-Timeout` header given to the Frontend when handling this request.

- Type: Histogram
- Available in: Core, Go, Java
- Tags: `namespace`, `task_queue`

### nexus_task_execution_failed

Handling of a Nexus Task resulted in an error. This includes any error returned from a user handler and unexpected
internal errors in the SDK.

- Type: Counter
- Available in: Go, Java
- Tags: `namespace`, `task_queue`, `nexus_service`, `nexus_operation`, `failure_reason`

Valid values for the `failure_reason` tag:

- `internal_sdk_error`: There was an unexpected internal error within the SDK while handling the Nexus task. Indicates a
  bug in the SDK.
- `handler_error_{TYPE}`: The user handler code returned a predefined error, as specified in the [Nexus spec](https://github.com/nexus-rpc/api/blob/main/SPEC.md#predefined-handler-errors).
  If the handler returns an unexpected error, the TYPE is set to `INTERNAL`.
- `timeout`: The user handler code did not return within the request timeout.
- `operation_failed`: The user handler code has indicated that the operation has failed. In Go, this maps to an
  `UnsuccessfulOperationError` with a `failed` state.
- `operation_canceled`: The user handler code has indicated that the operation has completed as canceled. In Go, this maps
  to an `UnsuccessfulOperationError` with a `canceled` state.

### nexus_task_execution_latency

Time to complete a Nexus Task, from the time the Nexus Task processing starts in the SDK to the time the user handler
completes.

- Type: Histogram
- Available in: Go, Java
- Tags: `namespace`, `task_queue`, `nexus_service`, `nexus_operation`

### nexus_task_endtoend_latency

Total latency of Nexus Tasks from the time the corresponding request hit the Frontend to after the SDK gets
acknowledgment from the server for task completion.

- Type: Histogram
- Available in: Go, Java
- Tags: `namespace`, `task_queue`, `nexus_service`, `nexus_operation`

### num_pollers

Current number of Worker Entities that are polling.

- Type: Gauge
- Available in: Core, Go, Java
- Tags: `namespace`, `poller_type`, `task_queue`

### poller_start

A Worker Entity poller was started.

- Type: Counter
- Available in: Go, Java
- Tags: `namespace`, `task_queue`

### request

Temporal Client made an RPC request.

- Type: Counter
- Available in: Core, Go, Java
- Tags: `namespace`, `operation`

### request_failure

Temporal Client made an RPC request that failed.
This number is included into the total `request` counter for RPC requests.

- Type: Counter
- Available in: Core, Go, Java
- Tags: `namespace`, `operation`

### request_latency

Latency of a Temporal Client gRPC request.

- Type: Histogram
- Available in: Core, Go, Java
- Tags: `namespace`, `operation`

### resource_slots_cpu_usage

CPU usage as a value between 0 and 100. As perceived by the resource-based slots tuner, if
enabled.

- Type: Gauge
- Available in: Core, Java

### resource_slots_mem_usage

Memory usage as a value between 0 and 100. As perceived by the resource-based slots tuner, if
enabled.

- Type: Gauge
- Available in: Core, Java

### sticky_cache_hit

A Workflow Task found a cached Workflow Execution to run against.

- Type: Counter
- Available in: Core, Go, Java
- Tags: `namespace`, `task_queue`

### sticky_cache_miss

A Workflow Task did not find a cached Workflow execution to run against.

- Type: Counter
- Available in: Core, Go, Java
- Tags: `namespace`, `task_queue`

### sticky_cache_size

Current cache size, expressed in number of Workflow Executions.

- Type: Gauge
- Available in: Core, Go, Java
- Tags: `namespace` (TypeScript, Java), `task_queue` (TypeScript)

### sticky_cache_total_forced_eviction

A Workflow Execution has been forced from the cache intentionally.

- Type: Counter
- Available in: Go, Java
- Tags: `namespace`, `task_queue`

### unregistered_activity_invocation

A request to spawn an Activity Execution is not registered with the Worker.

- Type: Counter
- Available in: Go,
- Tags: `activity_type`, `namespace`, `task_queue`, `workflow_type`

### worker_start

A Worker Entity has been registered, created, or started.

- Type: Counter
- Available in: Core, Go, Java
- Tags: `namespace`, `task_queue`, `worker_type`

### worker_task_slots_available

The total number of Workflow, Activity, Local Activity, or Nexus Task execution slots that are currently available.
Use the `worker_type` key to differentiate execution slots.
The Worker type specifies an ability to perform certain tasks.
For example, Workflow Workers execute Workflow Tasks, Activity Workers execute Activity Tasks, and so forth.

- Type: Gauge
- Available in: Core, Go, Java
- Tags: `namespace`, `task_queue`, `worker_type`

### worker_task_slots_used

The total number of Workflow, Activity, Local Activity, or Nexus Tasks execution slots in current use.
Use the `worker_type` key to differentiate execution slots.
The Worker type specifies an ability to perform certain tasks.
For example, Workflow Workers execute Workflow Tasks, Activity Workers execute Activity Tasks, and so forth.

- Type: Gauge
- Available in: Core, Go, Java
- Tags: `namespace`, `task_queue`, `worker_type`

### workflow_active_thread_count

Total amount of Workflow threads in the Worker Process.

- Type: Gauge
- Available in: Java

### workflow_cancelled

Workflow Execution ended because of a cancellation request.

- Type: Counter
- Available in: Core, Go, Java
- Tags: `namespace`, `task_queue`, `workflow_type`

### workflow_completed

A Workflow Execution completed successfully.

- Type: Counter
- Available in: Core, Go, Java
- Tags: `namespace`, `task_queue`, `workflow_type`

### workflow_continue_as_new

A Workflow ended with Continue-As-New.

- Type: Counter
- Available in: Core, Go, Java
- Tags: `namespace`, `task_queue`, `workflow_type`

### workflow_endtoend_latency

Total Workflow Execution time from schedule to completion for a single Workflow Run. (A retried Workflow Execution is a separate Run.)

- Type: Histogram
- Available in: Core, Go, Java
- Tags: `namespace`, `task_queue`, `workflow_type`

### workflow_failed

A Workflow Execution failed.

- Type: Counter
- Available in: Core, Go, Java
- Tags: `namespace`, `task_queue`, `workflow_type`

### workflow_task_execution_failed

A Workflow Task Execution failed.

- Type: Counter
- Available in: Core, Go, Java
- Tags: `namespace`, `task_queue`, `workflow_type`, `failure_reason`

Valid values for the `failure_reason` tag:

- `NonDeterminismError`: The Workflow Task failed due to a non-determinism error.
- `WorkflowError`: The Workflow Task failed for any other reason.

### workflow_task_execution_latency

Workflow Task Execution time.

- Type: Histogram
- Available in: Core, Go, Java
- Tags: `namespace`, `task_queue`, `workflow_type`

### workflow_task_queue_poll_empty

A Workflow Worker polled a Task Queue and timed out without picking up a Workflow Task.

- Type: Counter
- Available in: Core, Go, Java
- Tags: `namespace`, `task_queue`

### workflow_task_queue_poll_succeed

A Workflow Worker polled a Task Queue and successfully picked up a Workflow Task.

- Type: Counter
- Available in: Core, Go, Java
- Tags: `namespace`, `task_queue`

### workflow_task_replay_latency

Time to catch up on replaying a Workflow Task.

- Type: Histogram
- Available in: Core, Go, Java
- Tags: `namespace`, `task_queue`, `workflow_type`

### workflow_task_schedule_to_start_latency

The Schedule-To-Start time of a Workflow Task.

- Type: Histogram
- Available in: Core, Go, Java
- Tags: `namespace`, `task_queue`
