Skip to main content

Abuse Implementation Details

CreatedAuthor(s)Status
2022-04-20@jazzzDraft

Background & Motivation​

To implement the high level solutions outlined in https://github.com/xmtp-labs/hq/pull/422 more technical details are needed

This document should outline how the abuse prevention methods would be implemented

Goals / Non-goals​

  • Goals
    • Provide enough details to begin implementation of Abuse Mitigation
    • Outline core components/work streams.

Proposed Solution​

In Situ Implementation​

To increase development speed and reduce maintenance overhead, all abuse management approaches will be implemented directly on the nodes. Adding new infrastructure adds significant overhead and should be avoided at this time.

Authenticated Clients​

To submit messages a client must first authenticate via a side-channel. Authentication in this context means to prove its xmtp:identity to the nodes.

State/Action Tree​

stateDiagram-v2
    C : Transport Connected
    A : Transport Authenticated
    
    [*] -->C : [Connect]
    C --> C : [Read]
    C --> A : [Auth]
    C --> [*]: [Close]
    
    A --> A : [Read] 
    A --> A : [Write] 
    A --> [*] : [Close]

*actions not listed result in error

Notes​

  • Authentication will occur once for every client "connection" to a node.
  • The side-channel will be implemented via a new libp2p:protocol.
  • Client will submit the peerId signed by xmtp:identity
    • xmtp:node can verify by Signature->xmtp:identity == wallet-> xmtp:identity
      • Signing the peerId by the wallet address would add an additional signing request to the user and should be avoided
    • There is currently no method to retrieve a wallet address for a xmtp:identity, may need to have the clients include the intended wallet to simplify the lookup on the node
  • Node should keep mapping of peerId->walletAddress to be used in determining access/rate-limits
  • Users who send messages without first authenticating must receive an error

Example Sequence​

sequenceDiagram
    participant C as Client
    participant N as Node
    participant G as Alchemy

    C-xN: <Msg1>
    Note over C,N: Message Failed, not Authenticated
    C->>N: Auth:{wallet,Signature,Voucher}
    N->>G:  {Wallet}
    G-->>N: 
    Note over N,G: Wallet meets tx requirements 
    activate N
    C->>N: <Msg1>
    C->>N: <Msg2>
    C->>N: <Msg3>
    deactivate N

Block Chain State​

Maintaining a full Ethereum node is not sufficient on its own to satisfy our needs. The proof-of-work solution to account control requires a wallet->transaction mapping.

  • There is no RPC call for fetching transactions by wallet.
  • A dataset would need to be generated by listening for blocks and indexing wallet addresses. This is not an impassible obstacle, but adds more maintenance overhead

Alchemy offers a transaction api that will allow us to fetch the history for a given wallet. This seems like the easiest solution for getting blockchain data fast. If desired we can always migrate to our own eth-node in the future.

Accessing the alchemyApi from the nodes directly will keep the process as lightweight as possible. Results could be cached to speed up subsequent queries and lower alchemyApi usage.

Allow/Deny Lists​

Allow and Deny lists are expected to be quite small.

  • Data should be stored in a shared database.
  • Data should be cached on each node in order to speed up verification and increase fault tolerance.
  • The DB should be queried on a regular interval to check for new entries.
    • A 5min polling frequency will give an mean time to synchronization of 1m40s which should be sufficient.
    • Should this become a bottleneck in the future there are paths for improvement.
      • Use a pair of bloom filters to distribute lists -- 1M entries with no false positives requires ~4 MiB.
      • pushed allow/deny list updates to the nodes
  • A Retool application can be used for administering the Lists.

Rate Limiting​

Rate limiting will be implemented independently on each node using the Token Bucket Algorithm.

  • Rate-limits are tracked per wallet address

  • State will need to be tracked for each wallet address.

    • Given the memory footprint is small this can be kept in memory for the time being.
    • Persistence between restarts is not required, a user being able to send extra messages is not a critical issue

  • The client must be notified that the message was not sent due to a rate-limit violation

    • Currently the only mechanism implemented is to throw an error

  • There must be visibility into rate limiting events, either by existing logging or by an events database

Schema: walletAddress: lastUpdateTime, tokenCount parameters: tokensPerMinute: 1 bucketSize : 100

Note: with these parameters a user can send a message every 20 seconds for 50minutes before they get rate limited.

Rate limiting logic​

flowchart TB

auth((did peer\nAuth?))
allow((is on \nAllowList? ))
deny((is on \nDenyList? ))
rate((RateLimit \nexceeded? ))
S[Send Msg]
E[Error]

Start --> auth
auth --yes--> allow

allow --no-->deny
allow --yes-->S
deny --no--> rate

rate --no --> S
rate --yes-->E
deny--yes-->E
auth --no--> E

Plan​

  • Add basic libp2p:protocol for authentication

  • Change SDK to notify authentication channel when a libp2p:transport is established

  • Add rate-limiting via peerId -> Wallet lookup

  • Add Deny/Allow List infrastructure and synchronize with nodes

  • Expand rate limiting to consider Allow/Deny lists

  • Add blockchain data infra

  • Expand authentication to include blockchain information

Alternatives Considered / Prior Art?​

Voucher System​

A voucher is a placeholder term for an object which proves to a node that a client has rights to send messages. A voucher is useful to fully decouple nodes from blockchain data: A gatekeeper vouches for an Identity, and a node verifies the voucher.

While there have been many discussions of doing this in zero knowledge, there is also a possibility to accomplish the same effect with classic signature trust chains.

The signature is supplied to a node upon authentication, and the node verifies that it originated from a trusted source.

In this scheme the

  • Client contacts a gatekeeper server (eg: xmtp-labs)
  • Sever returns a voucher(signature) that the client meets requirements for the network
    • Vouchers must specify the number max chain length and an expiry date.
  • Client sends voucher to node during authentication, to prove it meets requirements
  • Client is not authenticated to send messages.

Signature is placed on the network so it can easily be retrieved. While not necessary to keep private, it would make sense to store it in the privateTopicKeyStore.

sequenceDiagram
 
 participant G as Voucher
 participant C as Client
 participant N as Node

 
C-xN: <Msg1>
Note over C,N: Message Failed, not Authenticated
C->>+G:  {Wallet}
G-->>-C: {voucher}
Note over C,G: Wallet meets tx requirements 
C->>N: Auth:{wallet,Signature,Voucher}
activate N
C->>N: <Msg1>
C->>N: <Msg2>
C->>N: <Msg3>
deactivate N

This concept adds significant complication that would require careful thought. In the interest of speed, a direct query approach is favored.

Risks?​

  • Due to the breadth of changes, there a high chance that some component/use-case is being overlooked.
    • Starting work now will increase visibility to those blind spots.
  • These approaches treat nodes as independent actors, in an effort to speed up development time. As nodes share little information between each other, abusers will be able to leverage this to their advantage. In the future building out shared data infrastructure for the nodes may be needed.
  • Currently cache implementations are ill-defined. Approaches to managing cache sizes and entry lifetimes have not been considered.
  • Keeping compatibility between SDK and xmtp-node-go will take some effort as substantial changes are made to the data flow of both repo's. Care is needed to ensure compatibility.

Questions​

  • How can calls to external APIs be limited from a DOS perspective? Currently every connection potentially results in a call to Alchemy endpoints. From an abuse lens this could result in high financial costs. Can requests/wallets be validated prior to calling alchemy and other 3rd party api's ?

Appendix​

…