Skip to content

ADR 001: Scalable Architecture & Batch Market Updates

Status: Accepted

Date: 2025-12-23

Deciders: Development Team

Technical Story: Issue: Scalable Architecture & Batch Market Updates for High-Traffic Scenarios

Executive Summary

This document outlines the scalable architecture design for the ClearPath & FairWins prediction market platform, with specific focus on batch processing capabilities, enhanced event structures, and efficient market discovery mechanisms to support high-traffic scenarios and explosive growth.

Context

The application is preparing for potential explosive growth and needs proactive scalability measures. Current system limitations include: - Individual transaction processing leading to high gas costs - No batch processing capabilities for markets or positions - Limited event indexing making market discovery inefficient - Sequential operations creating bottlenecks at scale

This ADR documents the architectural decisions to address these limitations and enable 100-1000x growth in concurrent markets.

Table of Contents

  1. Current System Analysis
  2. Scalability Challenges
  3. Batch Processing Architecture
  4. Enhanced Event Structure
  5. Market Discovery & Lookup API
  6. Gas Optimization Strategies
  7. Implementation Roadmap
  8. Performance Benchmarks

Current System Analysis

Architecture Overview

The current system consists of several key components:

  • FutarchyGovernor: Main coordination layer
  • ProposalRegistry: Proposal submission and management
  • ConditionalMarketFactory: Market deployment and resolution
  • PrivacyCoordinator: Encrypted position management
  • OracleResolver: Multi-stage oracle resolution
  • WelfareMetricRegistry: Welfare metrics tracking
  • RagequitModule: Minority protection

Current Limitations

1. Throughput Constraints

Operation Current Design Limitation Impact at Scale
Market Creation One-by-one deployment ~200k gas per market 5 markets = 1M gas
Position Updates Individual submissions ~100k gas per position 10 positions = 1M gas
Market Resolution Sequential processing ~150k gas per market High latency
Event Querying No pagination Full scan required O(n) complexity

2. State Storage Inefficiencies

  • Current: Each market stored in separate mapping slot
  • Issue: High storage costs (20,000 gas for new slot)
  • Impact: Linear cost increase with market count

3. Event Discovery Challenges

  • Current: Basic events without comprehensive indexing
  • Issue: Off-chain indexers must process all blocks
  • Impact: Slow market discovery, poor UX at scale

4. Memory-Bound Operations

  • Current: Loading full market state for each operation
  • Issue: Block gas limit constraints (30M gas on ETC)
  • Impact: Max ~150 markets processable in single transaction

Scalability Challenges

1. High-Traffic Scenarios

Expected Growth Pattern: - Launch: 10-50 markets/day - 6 months: 100-500 markets/day - 1 year: 1,000+ markets/day - Peak: 10,000+ concurrent markets

Challenges: - Block space competition - Gas price volatility - State bloat - Query performance degradation

2. Cost Efficiency Requirements

Target Metrics: - Market creation: <100k gas - Batch operations: <50k gas per item - Position updates: <30k gas per position - Query operations: <10k gas

3. User Experience Requirements

  • Market discovery: <2 seconds
  • Position updates: <5 seconds
  • Historical data: <3 seconds
  • Real-time updates: <1 second latency

Batch Processing Architecture

Design Principles

  1. Atomic Batch Execution: All-or-nothing batch processing
  2. Gas Optimization: Minimize storage operations
  3. Failure Isolation: Individual item failures don't block batch
  4. Transparent Logging: Comprehensive event emission

1. Batch Market Creation

Interface Design

struct MarketCreationParams {
    uint256 proposalId;
    address collateralToken;
    uint256 liquidityAmount;
    uint256 liquidityParameter;
    uint256 tradingPeriod;
}

function batchDeployMarkets(
    MarketCreationParams[] calldata params
) external onlyOwner returns (uint256[] memory marketIds);

Implementation Strategy

Optimizations: 1. Shared Storage Slots: Pack related data together 2. Calldata over Memory: Use calldata for input arrays 3. Pre-allocation: Reserve storage slots in batch 4. Event Batching: Single event for multiple markets

Gas Savings: - Single call: ~200k gas per market - Batch (10 markets): ~120k gas per market (40% savings) - Batch (50 markets): ~100k gas per market (50% savings)

Example Event Structure

event BatchMarketsCreated(
    uint256[] indexed marketIds,
    uint256[] proposalIds,
    uint256 batchTimestamp,
    uint256 totalMarketsCreated
);

event MarketCreatedInBatch(
    uint256 indexed marketId,
    uint256 indexed proposalId,
    uint256 indexed batchId,
    address passToken,
    address failToken,
    uint256 tradingEndTime
);

2. Batch Position Updates

Current Flow (Inefficient)

User 1 → Submit Position 1 → Process → Event
User 2 → Submit Position 2 → Process → Event
User 3 → Submit Position 3 → Process → Event
Total: 3 transactions, 3 × 100k = 300k gas

Optimized Flow (Batch Processing)

User 1, 2, 3 → Submit to Buffer
Coordinator → Process Batch [1,2,3] → Single Event
Total: 1 coordinator transaction, ~150k gas (50% savings)

Implementation Strategy

Privacy Coordinator Enhancement:

struct BatchPositionUpdate {
    uint256[] positionIds;
    bytes32[] commitments;
    bytes[] zkProofs;
    uint256 batchTimestamp;
}

function processBatchPositions(
    BatchPositionUpdate calldata batch
) external onlyCoordinator returns (bool);

Optimizations: 1. Commitment Batching: Process multiple commitments in single call 2. Proof Aggregation: Use batch ZK proof verification 3. Epoch Consolidation: Process full epochs atomically 4. Storage Packing: Pack position metadata efficiently

Gas Savings: - Single position: ~100k gas - Batch (10 positions): ~40k gas per position (60% savings) - Batch (50 positions): ~25k gas per position (75% savings)

3. Batch Market Resolution

Interface Design

struct MarketResolutionParams {
    uint256 marketId;
    uint256 passValue;
    uint256 failValue;
}

function batchResolveMarkets(
    MarketResolutionParams[] calldata params
) external onlyOwner returns (bool[] memory results);

Implementation Strategy

Features: 1. Parallel Processing: Resolve independent markets together 2. Failure Handling: Continue on individual failures 3. Result Tracking: Return success/failure for each market 4. Event Emission: Emit both batch and individual events

Gas Savings: - Single resolution: ~150k gas - Batch (10 resolutions): ~100k gas each (33% savings) - Batch (25 resolutions): ~80k gas each (47% savings)

Enhanced Event Structure

Design Principles

  1. Comprehensive Indexing: Index all query-relevant fields
  2. Hierarchical Events: Both summary and detail events
  3. Timestamp Precision: Block number + timestamp for all events
  4. Status Tracking: Emit events for all state transitions

1. Market Lifecycle Events

Enhanced Event Schema

// Market creation with full context
event MarketCreated(
    uint256 indexed marketId,
    uint256 indexed proposalId,
    address indexed collateralToken,
    address passToken,
    address failToken,
    uint256 tradingEndTime,
    uint256 liquidityParameter,
    uint256 createdAt,
    address creator
);

// Market status transitions
event MarketStatusChanged(
    uint256 indexed marketId,
    MarketStatus indexed previousStatus,
    MarketStatus indexed newStatus,
    uint256 changedAt,
    address changedBy
);

// Trading activity
event MarketTrade(
    uint256 indexed marketId,
    address indexed trader,
    bool indexed isPassToken,
    uint256 amount,
    uint256 price,
    uint256 timestamp,
    uint256 newPassPrice,
    uint256 newFailPrice
);

// Market resolution with outcomes
event MarketResolved(
    uint256 indexed marketId,
    uint256 indexed proposalId,
    uint256 passValue,
    uint256 failValue,
    bool indexed approved,
    uint256 resolvedAt,
    address resolver
);

// Market cancellation
event MarketCancelled(
    uint256 indexed marketId,
    uint256 indexed proposalId,
    string reason,
    uint256 cancelledAt,
    address cancelledBy
);

2. Position Management Events

// Position submission
event PositionSubmitted(
    uint256 indexed positionId,
    address indexed user,
    uint256 indexed marketId,
    bytes32 commitment,
    uint256 epoch,
    uint256 timestamp
);

// Batch position processing
event BatchPositionsProcessed(
    uint256 indexed batchId,
    uint256 indexed epochId,
    uint256[] positionIds,
    uint256 processedCount,
    uint256 timestamp
);

// Position settlement
event PositionSettled(
    uint256 indexed positionId,
    address indexed user,
    uint256 indexed marketId,
    uint256 payout,
    uint256 timestamp
);

3. Proposal Lifecycle Events

// Enhanced proposal submission
event ProposalSubmitted(
    uint256 indexed proposalId,
    address indexed proposer,
    string title,
    uint256 fundingAmount,
    address indexed fundingToken,
    uint256 welfareMetricId,
    uint256 bondAmount,
    uint256 submittedAt,
    uint256 reviewEndsAt,
    uint256 executionDeadline
);

// Phase transitions
event ProposalPhaseChanged(
    uint256 indexed proposalId,
    ProposalPhase indexed previousPhase,
    ProposalPhase indexed newPhase,
    uint256 changedAt
);

// Execution tracking
event ProposalExecuted(
    uint256 indexed proposalId,
    address indexed recipient,
    uint256 amount,
    address indexed fundingToken,
    uint256 executedAt,
    bytes32 txHash
);

4. Aggregate Statistics Events

// Daily statistics
event DailyStatistics(
    uint256 indexed date,
    uint256 marketsCreated,
    uint256 marketsResolved,
    uint256 totalVolume,
    uint256 uniqueTraders,
    uint256 totalPositions
);

// Market metrics snapshot
event MarketMetricsSnapshot(
    uint256 indexed marketId,
    uint256 indexed timestamp,
    uint256 totalVolume,
    uint256 totalLiquidity,
    uint256 passPrice,
    uint256 failPrice,
    uint256 uniqueTraders
);

Market Discovery & Lookup API

Design Goals

  1. Fast Queries: O(1) or O(log n) complexity
  2. Flexible Filtering: Multiple filter criteria
  3. Pagination Support: Handle large result sets
  4. Efficient Indexing: Minimize gas costs

1. On-Chain Query Functions

Market Discovery

/**
 * @notice Get active markets with pagination
 * @param offset Starting index
 * @param limit Maximum results to return
 * @return marketIds Array of market IDs
 * @return hasMore Whether more results exist
 */
function getActiveMarkets(
    uint256 offset,
    uint256 limit
) external view returns (
    uint256[] memory marketIds,
    bool hasMore
);

/**
 * @notice Get markets by status with filtering
 * @param status Market status to filter by
 * @param minLiquidity Minimum liquidity threshold
 * @param offset Starting index
 * @param limit Maximum results
 * @return markets Array of market data
 */
function getMarketsByStatus(
    MarketStatus status,
    uint256 minLiquidity,
    uint256 offset,
    uint256 limit
) external view returns (Market[] memory markets);

/**
 * @notice Get markets for a specific proposal
 * @param proposalId Proposal ID
 * @return marketId Associated market ID
 * @return market Market details
 */
function getMarketForProposal(
    uint256 proposalId
) external view returns (
    uint256 marketId,
    Market memory market
);

/**
 * @notice Get markets by date range
 * @param startTime Start timestamp
 * @param endTime End timestamp
 * @param offset Starting index
 * @param limit Maximum results
 * @return marketIds Array of market IDs in range
 */
function getMarketsByDateRange(
    uint256 startTime,
    uint256 endTime,
    uint256 offset,
    uint256 limit
) external view returns (uint256[] memory marketIds);

Market Statistics

/**
 * @notice Get market statistics
 * @param marketId Market ID
 * @return volume Total trading volume
 * @return liquidity Current liquidity
 * @return traderCount Number of unique traders
 * @return positionCount Total positions
 */
function getMarketStatistics(
    uint256 marketId
) external view returns (
    uint256 volume,
    uint256 liquidity,
    uint256 traderCount,
    uint256 positionCount
);

/**
 * @notice Get global platform statistics
 * @return totalMarkets Total markets created
 * @return activeMarkets Currently active markets
 * @return totalVolume Lifetime trading volume
 * @return totalTraders Unique trader count
 */
function getPlatformStatistics() external view returns (
    uint256 totalMarkets,
    uint256 activeMarkets,
    uint256 totalVolume,
    uint256 totalTraders
);

User Position Queries

/**
 * @notice Get user positions across all markets
 * @param user User address
 * @param offset Starting index
 * @param limit Maximum results
 * @return positionIds Array of position IDs
 * @return markets Associated market IDs
 */
function getUserPositions(
    address user,
    uint256 offset,
    uint256 limit
) external view returns (
    uint256[] memory positionIds,
    uint256[] memory markets
);

/**
 * @notice Get user position in specific market
 * @param user User address
 * @param marketId Market ID
 * @return hasPosition Whether user has position
 * @return positionId Position ID if exists
 * @return value Current position value
 */
function getUserMarketPosition(
    address user,
    uint256 marketId
) external view returns (
    bool hasPosition,
    uint256 positionId,
    uint256 value
);

2. Indexing Strategy

Storage Optimization

// Market status tracking for fast filtering
mapping(MarketStatus => uint256[]) private marketsByStatus;

// Time-based indexing for date range queries
mapping(uint256 => uint256[]) private marketsByDay; // day => market IDs

// User position tracking
mapping(address => uint256[]) private userPositionIds;
mapping(address => mapping(uint256 => uint256)) private userMarketPosition;

// Pagination metadata
struct PaginationInfo {
    uint256 totalCount;
    uint256 pageSize;
    uint256 lastUpdated;
}

mapping(MarketStatus => PaginationInfo) public statusPagination;

Efficient Updates

/**
 * @notice Update market index on status change
 * @dev Called internally on status transitions
 */
function _updateMarketIndex(
    uint256 marketId,
    MarketStatus oldStatus,
    MarketStatus newStatus
) internal {
    // Remove from old status index
    _removeFromStatusIndex(marketId, oldStatus);

    // Add to new status index
    _addToStatusIndex(marketId, newStatus);

    // Update time-based index
    uint256 day = block.timestamp / 1 days;
    marketsByDay[day].push(marketId);
}

3. Off-Chain Indexing Support

Event Indexing Schema

Recommended Database Schema:

-- Markets table
CREATE TABLE markets (
    market_id BIGINT PRIMARY KEY,
    proposal_id BIGINT,
    collateral_token VARCHAR(42),
    pass_token VARCHAR(42),
    fail_token VARCHAR(42),
    trading_end_time TIMESTAMP,
    liquidity_parameter NUMERIC,
    status VARCHAR(20),
    created_at TIMESTAMP,
    resolved_at TIMESTAMP,
    pass_value NUMERIC,
    fail_value NUMERIC
);

CREATE INDEX idx_markets_status ON markets(status);
CREATE INDEX idx_markets_created_at ON markets(created_at);
CREATE INDEX idx_markets_proposal ON markets(proposal_id);

-- Positions table
CREATE TABLE positions (
    position_id BIGINT PRIMARY KEY,
    user_address VARCHAR(42),
    market_id BIGINT,
    commitment BYTEA,
    epoch BIGINT,
    submitted_at TIMESTAMP,
    processed BOOLEAN,
    settled BOOLEAN
);

CREATE INDEX idx_positions_user ON positions(user_address);
CREATE INDEX idx_positions_market ON positions(market_id);
CREATE INDEX idx_positions_epoch ON positions(epoch);

-- Trades table
CREATE TABLE trades (
    trade_id SERIAL PRIMARY KEY,
    market_id BIGINT,
    trader_address VARCHAR(42),
    is_pass_token BOOLEAN,
    amount NUMERIC,
    price NUMERIC,
    timestamp TIMESTAMP,
    block_number BIGINT,
    tx_hash VARCHAR(66)
);

CREATE INDEX idx_trades_market ON trades(market_id);
CREATE INDEX idx_trades_trader ON trades(trader_address);
CREATE INDEX idx_trades_timestamp ON trades(timestamp);

-- Market statistics (materialized view)
CREATE MATERIALIZED VIEW market_stats AS
SELECT 
    market_id,
    COUNT(DISTINCT trader_address) as unique_traders,
    SUM(amount) as total_volume,
    MAX(timestamp) as last_trade_time
FROM trades
GROUP BY market_id;

GraphQL API Schema

type Market {
  id: ID!
  proposalId: Int!
  collateralToken: String!
  passToken: String!
  failToken: String!
  tradingEndTime: DateTime!
  liquidityParameter: BigInt!
  status: MarketStatus!
  createdAt: DateTime!
  resolvedAt: DateTime
  passValue: BigInt
  failValue: BigInt
  statistics: MarketStatistics!
  trades(first: Int, skip: Int): [Trade!]!
}

type MarketStatistics {
  totalVolume: BigInt!
  uniqueTraders: Int!
  currentPassPrice: BigInt!
  currentFailPrice: BigInt!
  totalLiquidity: BigInt!
}

enum MarketStatus {
  ACTIVE
  TRADING_ENDED
  RESOLVED
  CANCELLED
}

type Query {
  market(id: ID!): Market
  markets(
    status: MarketStatus
    minLiquidity: BigInt
    first: Int = 20
    skip: Int = 0
    orderBy: MarketOrderBy = CREATED_AT
    orderDirection: OrderDirection = DESC
  ): [Market!]!

  userPositions(
    user: String!
    first: Int = 20
    skip: Int = 0
  ): [Position!]!

  platformStatistics: PlatformStatistics!
}

type PlatformStatistics {
  totalMarkets: Int!
  activeMarkets: Int!
  totalVolume: BigInt!
  uniqueTraders: Int!
  last24hVolume: BigInt!
}

REST API Specification

openapi: 3.0.0
info:
  title: Prediction Market Lookup API
  version: 1.0.0
  description: RESTful API for market discovery and querying

paths:
  /markets:
    get:
      summary: List markets with filtering and pagination
      parameters:
        - name: status
          in: query
          schema:
            type: string
            enum: [active, trading_ended, resolved, cancelled]
        - name: min_liquidity
          in: query
          schema:
            type: integer
        - name: offset
          in: query
          schema:
            type: integer
            default: 0
        - name: limit
          in: query
          schema:
            type: integer
            default: 20
            maximum: 100
      responses:
        '200':
          description: List of markets
          content:
            application/json:
              schema:
                type: object
                properties:
                  markets:
                    type: array
                    items:
                      $ref: '#/components/schemas/Market'
                  pagination:
                    $ref: '#/components/schemas/Pagination'

  /markets/{marketId}:
    get:
      summary: Get market details
      parameters:
        - name: marketId
          in: path
          required: true
          schema:
            type: integer
      responses:
        '200':
          description: Market details
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Market'

  /markets/{marketId}/statistics:
    get:
      summary: Get market statistics
      parameters:
        - name: marketId
          in: path
          required: true
          schema:
            type: integer
      responses:
        '200':
          description: Market statistics
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/MarketStatistics'

  /users/{address}/positions:
    get:
      summary: Get user positions
      parameters:
        - name: address
          in: path
          required: true
          schema:
            type: string
        - name: offset
          in: query
          schema:
            type: integer
            default: 0
        - name: limit
          in: query
          schema:
            type: integer
            default: 20
      responses:
        '200':
          description: User positions
          content:
            application/json:
              schema:
                type: object
                properties:
                  positions:
                    type: array
                    items:
                      $ref: '#/components/schemas/Position'
                  pagination:
                    $ref: '#/components/schemas/Pagination'

components:
  schemas:
    Market:
      type: object
      properties:
        id:
          type: integer
        proposalId:
          type: integer
        collateralToken:
          type: string
        passToken:
          type: string
        failToken:
          type: string
        tradingEndTime:
          type: string
          format: date-time
        status:
          type: string
        liquidityParameter:
          type: string
        statistics:
          $ref: '#/components/schemas/MarketStatistics'

    MarketStatistics:
      type: object
      properties:
        totalVolume:
          type: string
        uniqueTraders:
          type: integer
        currentPassPrice:
          type: string
        currentFailPrice:
          type: string
        totalLiquidity:
          type: string

    Position:
      type: object
      properties:
        id:
          type: integer
        marketId:
          type: integer
        userAddress:
          type: string
        commitment:
          type: string
        submittedAt:
          type: string
          format: date-time

    Pagination:
      type: object
      properties:
        offset:
          type: integer
        limit:
          type: integer
        total:
          type: integer
        hasMore:
          type: boolean

Gas Optimization Strategies

1. Storage Optimization

Packed Structs

// Before: 5 storage slots
struct Market {
    uint256 proposalId;        // slot 0
    uint256 tradingEndTime;    // slot 1
    uint256 liquidityParameter; // slot 2
    address passToken;         // slot 3
    address failToken;         // slot 4
}
// Cost: 5 × 20,000 = 100,000 gas

// After: 3 storage slots
struct Market {
    uint256 proposalId;        // slot 0
    uint256 tradingEndTime;    // slot 1
    uint256 liquidityParameter; // slot 2 (first 128 bits)
    address passToken;         // slot 2 (96 bits)
    address failToken;         // slot 3 (160 bits)
    bool resolved;             // slot 3 (8 bits)
}
// Cost: 3 × 20,000 = 60,000 gas
// Savings: 40,000 gas (40%)

Use Events for Historical Data

// Don't store historical trades on-chain
// Instead, emit events and index off-chain
event MarketTrade(
    uint256 indexed marketId,
    address indexed trader,
    uint256 amount,
    uint256 price,
    uint256 timestamp
);

// Only store current state
mapping(uint256 => uint256) public currentPassPrice;
mapping(uint256 => uint256) public currentFailPrice;

2. Computation Optimization

Avoid Repeated Calculations

// Before: Recalculate on every access
function getMarketPrice(uint256 marketId) public view returns (uint256) {
    uint256 passQty = passTokens[marketId];
    uint256 failQty = failTokens[marketId];
    uint256 beta = liquidityParams[marketId];
    // Complex LMSR calculation...
    return calculatePrice(passQty, failQty, beta);
}

// After: Cache calculated values
mapping(uint256 => uint256) private cachedPassPrices;
mapping(uint256 => uint256) private cachedFailPrices;

function updateMarketPrice(uint256 marketId) internal {
    (uint256 passPrice, uint256 failPrice) = calculatePrices(marketId);
    cachedPassPrices[marketId] = passPrice;
    cachedFailPrices[marketId] = failPrice;
}

Use Unchecked Math Where Safe

// Safe to use unchecked when overflow impossible
function _addToArray(uint256[] storage arr, uint256 value) internal {
    uint256 len = arr.length;
    arr.push(value);
    unchecked {
        // Length can never overflow in practice
        assert(arr.length == len + 1);
    }
}

3. Calldata vs Memory

// Use calldata for read-only arrays (saves ~3 gas per word)
function batchProcess(
    uint256[] calldata ids,  // Use calldata
    uint256[] calldata values // Use calldata
) external {
    // Process without copying to memory
    for (uint256 i = 0; i < ids.length; ) {
        _process(ids[i], values[i]);
        unchecked { ++i; }
    }
}

4. Short-Circuit Evaluation

// Order checks from cheapest to most expensive
function validateMarket(uint256 marketId) public view returns (bool) {
    // Check storage first (cheapest)
    if (marketId >= marketCount) return false;

    // Then check mapping
    Market storage market = markets[marketId];
    if (market.status != MarketStatus.Active) return false;

    // Finally check timestamp (external call potential)
    if (block.timestamp < market.tradingEndTime) return false;

    return true;
}

5. Batch Storage Updates

// Instead of multiple SSTORE operations
function updateMarketsSequential(uint256[] calldata ids) external {
    for (uint256 i = 0; i < ids.length; i++) {
        markets[ids[i]].status = MarketStatus.Resolved; // Multiple SSTOREs
    }
}

// Use single SSTORE by updating memory first
function updateMarketsBatch(uint256[] calldata ids) external {
    for (uint256 i = 0; i < ids.length; i++) {
        Market storage market = markets[ids[i]];
        market.status = MarketStatus.Resolved;
        market.resolvedAt = block.timestamp;
        market.passValue = passValues[i];
        market.failValue = failValues[i];
        // All updates in single storage slot access pattern
    }
}

Implementation Roadmap

Phase 1: Foundation (Weeks 1-2)

Week 1: Contract Enhancements

  • Add batch market creation to ConditionalMarketFactory
  • Implement storage packing for Market struct
  • Add comprehensive event structure
  • Create pagination helper functions
  • Write unit tests for batch operations

Week 2: Indexing Infrastructure

  • Implement status-based indexing
  • Add time-based market tracking
  • Create user position tracking
  • Add pagination metadata
  • Write integration tests

Deliverables: - Updated ConditionalMarketFactory contract - Enhanced event definitions - Test coverage >90% - Gas optimization report

Phase 2: Batch Processing (Weeks 3-4)

Week 3: Position Batching

  • Enhance PrivacyCoordinator for batch processing
  • Implement batch position submission
  • Add batch ZK proof verification
  • Create epoch consolidation logic
  • Write fuzz tests

Week 4: Resolution Batching

  • Add batch resolution to ConditionalMarketFactory
  • Implement failure handling
  • Add batch events
  • Create monitoring tools
  • Performance testing

Deliverables: - Batch-enabled PrivacyCoordinator - Batch resolution functionality - Performance benchmarks - Gas comparison analysis

Phase 3: Query API (Weeks 5-6)

Week 5: On-Chain Queries

  • Implement getActiveMarkets with pagination
  • Add getMarketsByStatus filtering
  • Create market statistics functions
  • Add user position queries
  • Write view function tests

Week 6: Off-Chain Support

  • Design database schema
  • Create GraphQL API specification
  • Document REST API endpoints
  • Build example indexer
  • Write integration guide

Deliverables: - Complete query function library - Database schema documentation - API specifications - Example indexer implementation

Phase 4: Optimization & Testing (Weeks 7-8)

Week 7: Gas Optimization

  • Profile gas usage across operations
  • Implement identified optimizations
  • Benchmark before/after
  • Document savings
  • Update cost projections

Week 8: Security & Testing

  • Security audit preparation
  • Comprehensive test suite
  • Fuzzing campaign
  • Load testing
  • Documentation review

Deliverables: - Optimized contract suite - Security audit report - Load test results - Complete documentation

Phase 5: Deployment & Monitoring (Weeks 9-10)

Week 9: Testnet Deployment

  • Deploy to Mordor testnet
  • Run integration tests
  • Monitor performance
  • Gather community feedback
  • Fix identified issues

Week 10: Production Preparation

  • Mainnet deployment plan
  • Monitoring dashboard
  • Alerting setup
  • Documentation finalization
  • Training materials

Deliverables: - Testnet deployment - Monitoring infrastructure - Deployment documentation - Operator training

Performance Benchmarks

Expected Performance Improvements

Gas Costs

Operation Current Optimized Savings
Create 1 Market 200k 200k 0%
Create 10 Markets 2.0M 1.2M 40%
Create 50 Markets 10M 5.0M 50%
Update 1 Position 100k 100k 0%
Update 10 Positions 1.0M 400k 60%
Update 50 Positions 5.0M 1.25M 75%
Resolve 1 Market 150k 150k 0%
Resolve 10 Markets 1.5M 1.0M 33%
Resolve 25 Markets 3.75M 2.0M 47%

Query Performance

Query Type Current Optimized Improvement
Find Active Markets O(n) scan O(1) lookup 100x faster
Market by ID O(1) O(1) Same
Markets by Status O(n) scan O(m) where m=results 10-100x faster
User Positions O(n) scan O(k) where k=user positions 50-500x faster
Date Range Query Not supported O(d) where d=days New feature

Scalability Metrics

Metric Current Limit Optimized Limit Improvement
Markets per Day 50 500+ 10x
Concurrent Markets 100 10,000+ 100x
Positions per Epoch 50 1,000+ 20x
Query Response Time 5-10s <1s 5-10x
Indexer Sync Time 1 hour 5 minutes 12x

Load Testing Scenarios

Scenario 1: High Market Creation Rate

Test: Create 100 markets in rapid succession
Expected: 
- Gas: ~10M total (100k per market)
- Time: ~20 seconds (5 markets/second)
- Success Rate: >99%

Scenario 2: Heavy Trading Load

Test: 1000 position updates across 100 markets
Expected:
- Gas: ~25M total (25k per position)
- Time: ~3 minutes (batches of 50)
- Success Rate: >99.9%

Scenario 3: Mass Market Resolution

Test: Resolve 50 markets simultaneously
Expected:
- Gas: ~2M total (40k per market)
- Time: ~10 seconds
- Success Rate: 100%

Scenario 4: Query Performance

Test: 10,000 concurrent market lookups
Expected:
- Response Time: <100ms per query
- Throughput: >100 queries/second
- Cache Hit Rate: >90%

Risk Mitigation

Technical Risks

Risk Impact Likelihood Mitigation
Batch operation failure High Medium Implement failure isolation
Gas limit exceeded High Low Enforce batch size limits
Event indexing lag Medium Medium Implement backfill mechanism
Query performance degradation High Medium Add caching layer
Storage bloat Medium High Use events for historical data

Operational Risks

Risk Impact Likelihood Mitigation
Network congestion Medium High Adaptive gas pricing
Coordinator downtime High Low Automated failover
Database corruption High Low Regular backups, replication
API rate limiting Medium Medium CDN + caching

Monitoring & Observability

Key Metrics to Track

Smart Contract Metrics

  • Gas usage per operation type
  • Batch sizes and success rates
  • Failed transaction reasons
  • Storage utilization
  • Event emission rates

API Metrics

  • Query response times (p50, p95, p99)
  • Query throughput
  • Error rates
  • Cache hit rates
  • Database query performance

Business Metrics

  • Markets created per day
  • Active markets count
  • Trading volume
  • Unique users
  • Position update rate

Alerting Thresholds

alerts:
  - name: high_gas_usage
    condition: avg_gas > 200k
    severity: warning

  - name: batch_failure_rate
    condition: failure_rate > 5%
    severity: critical

  - name: slow_queries
    condition: p95_response_time > 2s
    severity: warning

  - name: indexer_lag
    condition: blocks_behind > 100
    severity: critical

  - name: storage_growth
    condition: daily_growth > 1GB
    severity: warning

Conclusion

This scalability architecture provides a comprehensive roadmap for supporting high-traffic scenarios while maintaining cost efficiency and user experience. The batch processing mechanisms, enhanced event structures, and efficient query APIs will enable the platform to scale from hundreds to thousands of concurrent markets without degradation.

Key Benefits

  1. 40-75% gas savings through batch operations
  2. 10-100x faster queries with proper indexing
  3. 100x capacity increase in concurrent markets
  4. Comprehensive monitoring for proactive management
  5. Future-proof architecture for continued growth

Next Steps

  1. Review and approve architecture design
  2. Prioritize implementation phases
  3. Allocate development resources
  4. Begin Phase 1 implementation
  5. Set up continuous monitoring

For questions or clarifications, please refer to the specific sections above or contact the development team.