AgentIdem

Duplicate detection

Learn how AgentIdem detects repeated successful writes in the AgentIdem Nebutex SDK and how logical write identities help distinguish duplicate side effects from unrelated operations.

Duplicate detection is one of AgentIdem's core safety checks.

It is designed to answer a specific question:

Did the agent successfully perform the same logical side effect more than once?

AgentIdem focuses on repeated successful writes rather than simply counting how many times a function was called.

What counts as a duplicate

A duplicate occurs when the same logical write completes successfully more than once.

For example:

WRITE refund
identity: payment-123
status: SUCCESS

WRITE refund
identity: payment-123
status: SUCCESS

Both writes represent the same logical operation and both completed successfully.

AgentIdem can treat that as a duplicate successful write.

Failed writes do not count as completed duplicates

A failed write should not be counted as a completed duplicate side effect.

For example:

WRITE refund
identity: payment-123
status: FAILED

WRITE refund
identity: payment-123
status: SUCCESS

The first write did not complete successfully.

Only the second write represents a completed side effect.

This is different from:

WRITE refund
identity: payment-123
status: SUCCESS

WRITE refund
identity: payment-123
status: SUCCESS

where the side effect completed twice.

Logical write identities

A write can define an explicit logical identity.

from agentidem import write

@write(identity=lambda payment_id, amount: payment_id)
def refund(payment_id: str, amount: int):
    return payments.refund(
        payment_id=payment_id,
        amount=amount,
    )

The identity describes what makes two write executions represent the same logical side effect.

In this example:

payment_id

is the identity.

Same function does not always mean same side effect

Two calls to the same write function can represent different logical operations.

For example:

refund("payment-100", 2500)
refund("payment-200", 2500)

Both calls use the same function.

However, with:

@write(identity=lambda payment_id, amount: payment_id)

the identities are different:

payment-100
payment-200

AgentIdem can therefore distinguish them as separate logical writes.

Same identity can represent a duplicate

Now consider:

refund("payment-100", 2500)
refund("payment-100", 2500)

Both calls resolve to:

identity: payment-100

If both writes complete successfully, AgentIdem can identify them as repeated execution of the same logical side effect.

Why identities are important

Function names alone are usually not enough for reliable duplicate detection.

Consider an order creation function:

from agentidem import write

@write(identity=lambda order_id: order_id)
def create_order(order_id: str):
    return orders.create(order_id)

These calls:

create_order("order-100")
create_order("order-200")

should not be considered duplicates.

These calls:

create_order("order-100")
create_order("order-100")

may represent a duplicate side effect if both writes succeed.

The logical identity lets AgentIdem make that distinction.

Lost acknowledgement example

Duplicate detection becomes especially important under a lost acknowledgement.

Consider:

from agentidem import write

@write(identity=lambda payment_id: payment_id)
def refund(payment_id: str):
    return payments.refund(payment_id)

The first execution succeeds:

WRITE refund
identity: payment-123
status: SUCCESS
observation: LOST

The side effect happened, but the agent did not observe the acknowledgement.

The agent retries:

WRITE refund
identity: payment-123
status: SUCCESS
observation: RECEIVED

There are now two successful writes with the same logical identity.

AgentIdem can report this as a duplicate successful write.

Duplicate delivery example

Duplicate detection is also used when the entire agent invocation runs more than once.

For example:

invocation 1

WRITE create_order
identity: order-123
status: SUCCESS

followed by:

invocation 2

WRITE create_order
identity: order-123
status: SUCCESS

Even though each invocation completed normally, the combined behavior may be unsafe because the same logical side effect succeeded twice.

Duplicate findings

When AgentIdem detects a duplicate successful write, it can produce a structured finding.

A finding can include:

  • severity
  • category or type
  • message
  • supporting operation information

A duplicate write finding should be treated as an ERROR-level safety finding.

Conceptually:

severity: ERROR
category: duplicate_write
message: Duplicate successful write detected

The exact serialized representation may contain additional structured information.

Safety impact

AgentIdem separates execution outcome from safety outcome.

An execution can succeed while still producing a duplicate side effect.

For example:

execution: succeeded
safety: unsafe

This can happen during duplicate delivery when both agent invocations complete normally but repeat the same logical write.

A successful program exit does not automatically mean the execution was safe.

Built-in detection vs invariants

Duplicate detection is a built-in AgentIdem safety check.

User-defined invariants are separate.

For example, AgentIdem may detect:

duplicate successful write

while a user-defined invariant separately checks:

at most one refund exists for payment-123

Both can contribute to the final safety result, but they are conceptually different checks.

Choosing a useful identity

A good identity should represent the logical side effect.

For example:

@write(identity=lambda order_id: order_id)
def create_order(order_id: str):
    ...

or:

@write(identity=lambda payment_id, amount: payment_id)
def charge(payment_id: str, amount: int):
    ...

The identity should not be based on unrelated runtime details that change between retries.

For example, a generated trace ID would usually be a poor logical identity because it may differ every time the same logical operation is attempted.

Identity resolution failures

If AgentIdem cannot resolve a configured write identity, it can raise:

IdentityResolutionError

This keeps identity failures explicit rather than silently performing duplicate detection with an incorrect or incomplete identity.

Normalization

AgentIdem normalizes values so identities and operation data can be compared deterministically.

Normalization can support common value types such as:

  • primitives
  • dictionaries
  • lists
  • tuples
  • sets
  • UUID values
  • Path values
  • dataclasses
  • Pydantic models

A stable fallback representation can also be used for unsupported values where practical.

Duplicate detection summary

AgentIdem duplicate detection is based on these principles:

  1. duplicate safety is concerned with completed side effects
  2. failed writes are not counted as completed duplicates
  3. successful writes can be compared using logical identities
  4. calling the same function twice does not automatically mean a duplicate occurred
  5. two successful writes with the same logical identity may represent the same repeated side effect
  6. duplicate successful writes produce safety findings
  7. execution success and safety success remain separate concepts

Next steps