Account update visualizer

This topic is to discuss the proposal submitted by @YashBit .Please see below for the details of the proposal and discussion.

6th Jan ,2025
Current status: Funded
Funding Note: Account updates are difficult to understand for developers working on Mina. This project is aimed at developers to improve their experience of building on Mina.

16th Oct ,2024
Current status: Under Consideration.
Opened for community discussion on 16th Oct, 2024.

2 Likes

Visualize Account Update (Issue #14)

Project Background

The Mina Protocol enables complex transactions resulting in multiple account updates, including token transfers, state changes, and delegation modifications. Currently, developers can view these updates in JSON format using the toPretty() function. However, as transactions become more intricate, the JSON output becomes difficult to interpret, especially for transactions involving multiple accounts or complex smart contract interactions.

This project aims to develop a VSCode extension that integrates with JSON Crack to create a powerful transaction visualization tool that will:

  • Generate and visually represent account update JSONs within VSCode
  • Allow developers to easily understand transaction structures without network deployment
  • Facilitate debugging of complex smart contract interactions
  • Leverage JSON Crack’s advanced visualization capabilities

By creating a more intuitive way to visualize and comprehend transaction structures directly within the development environment, this tool will significantly enhance the development experience on the Mina Protocol, addressing a key need identified by the developer community.

Proposal Overview

Problem

Complex transactions in the Mina Protocol result in JSON outputs that are difficult to interpret, especially for transactions involving multiple accounts or intricate smart contract interactions. This hinders developers’ ability to understand and debug their transactions efficiently within their development environment.

Solution

Develop a VSCode extension that integrates with JSON Crack to visually represent account updates and transaction structures. This tool will transform complex JSON outputs into intuitive, interactive graph visualizations directly within VSCode.

Impact

This proposal will significantly enhance the Mina ecosystem by:

  1. Improving developer productivity and reducing debugging time by providing in-editor visualization
  2. Lowering the barrier to entry for new developers in the Mina ecosystem
  3. Enabling more complex and innovative applications by making intricate transactions easier to understand and implement
  4. Leveraging the power of JSON Crack to provide advanced visualization features for Mina transactions

Audience

The primary target audience includes:

  1. Mina Protocol developers working on smart contracts and complex transactions
  2. New developers entering the Mina ecosystem
  3. Blockchain explorers and wallet developers in the Mina ecosystem who can adapt this tool for their needs

Architecture & Design

Detailed Design/Architecture

The proposed VSCode extension for Mina Transaction Visualization will integrate with JSON Crack and focus on the following components:

  1. State Analyzer:

    • Code Parser: Extracts account states from the code using the VSCode extension.
    • State Transition Detector: Identifies changes between states. We envision it more than just a debugger. The extension would attempt to simulate the txns/account updates while the test runs and/or while the code is being written and print the transitions next to the code with an option to double click and drill down further if needed
    • Relationship Builder: Creates relationships between state variables
  2. JSON Crack Integration:

    • Data Formatter: Converts analyzed data into JSON Crack compatible format
    • Visualization Engine: Generates the interactive graph using JSON Crack
    • Export Functions: Allows exporting the visualization as images or data. All attempts would be made to cover probable dev environments.
  3. VSCode Extension Interface:

    • WebView: Displays the JSON Crack visualization within VSCode
    • User Interaction Handlers: Manages code selection, analysis triggering, and graph interaction.

Diagram 1: High Level Design of Information Flow

Diagram 2: Example UI

Vision

Our vision for this project is to create a seamless, in-editor transaction visualization tool that significantly improves the Mina development experience. Key aspects of this vision include:

  1. Seamless Integration: Providing visualization capabilities directly within VSCode, where developers spend most of their time.
  2. Enhanced Detail: Offering comprehensive and easily digestible information about each AccountUpdate and its components.
  3. Interactivity: Leveraging JSON Crack’s interactive features for a dynamic exploration of transaction structures.
  4. Export Options: Enabling developers to easily share and incorporate transaction visualizations into their documentation and discussions.

By realizing this vision, we aim to improve the developer experience and contribute to the growth of the Mina ecosystem. This VSCode extension will serve as a powerful tool in the Mina development toolkit.

Deliverables

Mid-Point Milestones (1 month):

  1. State Analyzer:

    • Completed Code Parser for extracting account states
    • Implemented State Transition Detector
    • Basic Relationship Builder functionality
  2. VSCode Extension (Basic Version):

    • Functional VSCode extension with WebView
    • Ability to select code and trigger analysis
    • Basic integration with JSON Crack for visualization
  3. JSON Crack Integration (Partial):

    • Data Formatter for converting to JSON Crack format
    • Basic interactive graph generation within VSCode

Project Timeline: 2 Months

Week 1-2:

  • Develop State Analyzer components
  • Set up VSCode extension project structure

Week 3-4:

  • Implement basic VSCode extension functionality
  • Begin JSON Crack integration
  • Mid-point review and adjustments

Week 5-6:

  • Enhance JSON Crack integration with advanced features
  • Implement export functionalities
  • Comprehensive testing and bug fixes

Final 2 weeks:

  • Final polishing of UI/UX within VSCode
  • Documentation writing
  • Preparation for VSCode Marketplace deployment

Budget Requested: 45,000 MINA

Budget Breakdown:

  1. Development Costs: 35,000 MINA

    • State Analyzer Development: 12,000 MINA
    • VSCode Extension Development: 13,000 MINA
    • JSON Crack Integration: 10,000 MINA
  2. Testing and Quality Assurance: 5,000 MINA

  3. Documentation and Deployment: 3,000 MINA

  4. Project Management: 2,000 MINA

Wallet Address:

B62qrF8xMKPPQMCwK2J7Ey47CGy1bKvaKhYNuYURN8qVohr1NJcnhRj

Team Info

Proposer Github: If you have Github profiles that are relevant to this proposal, list them here.

Yash Bharti: YashBit (yash_b) Β· GitHub

Proposer Experience: If you have relevant experience building zkApps or other dApps, provide that information here. If available, include links.

  1. PSE - Ethereum Foundation Grantee
  2. Internet Computer Grantee
  3. Distributed Systems, Web Dev, ZK
  4. Rust, C++, Python, Next JS

Team Members:

  1. Krishanu Dhar: ronykris (krish) Β· GitHub
  2. Rute Figueiredo: rutefig (Rute Figueiredo) Β· GitHub

Risks & Mitigations

  1. Technical Complexity: Enhanced parsing of AccountUpdate objects may be more complex than anticipated.

    Mitigation: Begin with thorough analysis of existing code. Prioritize core functionalities and prototype early.

  2. Performance Issues: Increased detail in visualization may cause performance bottlenecks.

    Mitigation: Implement progressive rendering and lazy loading. Conduct regular performance testing.

  3. Compatibility with Future Mina Updates: Changes to Mina’s transaction structure could break the visualizer.

    Mitigation: Design a modular core engine. Maintain communication with Mina core team. Implement extensive unit testing.

  4. Scope Creep: Risk of exceeding the 1.5-month timeline due to feature expansion.

    Mitigation: Clearly define and prioritize features upfront. Use agile methodologies. Be prepared to adjust scope to meet timeline.

By anticipating these risks and implementing mitigations, we aim to ensure smooth development and delivery of the Enhanced Mina Transaction Visualizer within the proposed timeline and budget.

3 Likes

I find this proposal quite interesting.

It has two main benefits from my perspective:

  1. The proposal itself: It should make debugging zkApps much easier by allowing easy identification of state changes. It would be great if events, actions, and possibly some chain information like block height were also displayed.

  2. Potential team acquisition: With this proposal implemented, Mina can acquire a team that understands both Mina’s architecture and the structure of VS Code. This team would be able to create additional modules for VS Code, with potential plugins such as unprovable code detection, debugging tools, and more.

The only concern is the team’s experience in development using o1js. If they lack this experience, a mentor with expertise would be necessary, as Mina has some unique features that can take time to master, potentially affecting the proposal’s timeline.

2 Likes

I am summarising the feedback we have received from developers in the community.

Technical information

The AccountUpdates has different representations depending on where and how we view them
tx object (not JSON) after Mina.transaction is a tree, and there is NO function that serialize it fully. This function should be developed, probably as o1js PR in case there will be private fields not accessible or, other way, access the fields by using any, bypassing o1js controls.

  • json received as tx.toJSON()
  • json received after sending the tx
  • json received after the tx is included into the block
  • json in usual node
  • json in archive node (miss some fields due to the bug in archive node)
  • json in blockberry (mix of two above).

It is important to get errors when sending the tx and link the errors to the specific AccountUpdates.All of the above representations except first are flat arrays, the first is tree with child updates and is more interesting to analyze.It is not enough to visualize the JSON produced by o1js by default.

It would be great to support several standard contracts to mark the type of contracts and accounts in AccountUpdates (like FungibleToken and FungibleTokenAdmin) in visualizations.

State Analyzer:

Code Parser: Extracts account states from the code in VSCode
State Transition Detector: Identifies changes between states
Relationship Builder: Creates relationships between state variables

AccountUpdate building is deep in the o1js code and you wont be able to rebuild AccountUpdates just by analyzing the contract code, you need actually run Mina.transaction to get the forest of AccountUpdates.

Recommendation

It would be better if the team spends some time working with account udpdates to understand the current issue and update the proposal as the solution aimed for VS code might not be feasible. It might be easier to start with balance transfer states & move to zkApp states. We feel the solution improving the JSON output of the update is more helpful than a VS code integration.

2 Likes

After discussing with the team, this proposal is on hold and the team will be researching designs on the solution in the coming weeks.

Below is the revised proposal based on the feedback.

Visualize Account Update (Issue #14)

Project Background

The Mina Protocol enables complex transactions resulting in multiple account updates, including token transfers, state changes, and delegation modifications. Currently, developers can view these updates in JSON format using the toPretty() function. However, as transactions become more intricate, the JSON output becomes difficult to interpret, especially for transactions involving multiple accounts or complex smart contract interactions.

This project aims to create a NPM Package to recursively traverse the transaction tree and take a snapshot to create a powerful visualization of the state transformations. The tool will aim to:

  • Generate and visually represent state changes across account updates
  • Allow developers to easily decipher txn structures
  • Facilitate debugging of complex smart contract interactions

By creating a more intuitive way to visualize and comprehend transaction structures directly within the development environment, this tool will significantly enhance the development experience on the Mina Protocol, addressing a key need identified by the developer community.

Proposal Overview

Problem

Complex transactions in the Mina Protocol result in txn and account update trees that are difficult to interpret, especially for transactions involving multiple accounts or intricate smart contract interactions. This hinders developers’ ability to understand and debug their transactions efficiently within the development environment.

Solution

Develop a NPM Package that can be invoked on txn and accounts update (AU) tree to recursively traverse it and decipher the salient data in the nodes and leafs. Also, create a visualisation of the state transitions and call out the associated updates to the tree.

Impact

This proposal will significantly enhance the Mina ecosystem by:

  1. Improving developer productivity and reducing debugging time
  2. Lowering the barrier to entry for new dev in the Mina ecosystem
  3. Enabling more complex and innovative applications by making intricate transactions easier to understand and implement
  4. Leveraging open source tools to provide advanced visualization features

Audience

The primary target audience includes:

  1. Mina Protocol developers working on complex smart contracts
  2. New developers onboarding the Mina ecosystem
  3. Blockchain explorers and wallet developers in the ecosystem

Architecture & Design

Detailed Design/Architecture

Listed below are the components of the Mina Txn Visualiser

  1. State Analyser:

    • Recursively traverses the Account Update trees and takes a snapshot by serialising it.
    • Identifies and records changes between states.
    • Generates a relationship graph across the account update trees.
    • Records important state variables and their transition.
  2. Visualiser:

    • Format the result into a meaningful graph for easy visualisation of state transition and relationship.
    • Summarise the transaction flow in markdown and/or ascii format

Below is a high level class diagram of the state analyser and visualiser. (This may go through changes during the actual implementation)

Vision

Our vision for this project is to create a seamless, transaction visualization tool that significantly improves the Mina development experience. Key aspects of this vision include:

  1. Seamless Integration: Providing visualization capabilities directly within an IDE, where developers spend most of their time.
  2. Enhanced Detail: Offering comprehensive and easily digestible information about each AccountUpdate and its components.
  3. Distribution: Easily accessible and available as open source software.
  4. Export Options: Enabling developers to easily share and incorporate transaction visualizations into their documentation and discussions.

By realizing this vision, we aim to improve the developer experience and contribute to the growth of the Mina ecosystem. This package will serve as a powerful tool in the Mina development toolkit.

Deliverables

Mid-Point Milestones (4 weeks):

  1. State Analyzer:

    • Completed code parser for extracting account states
    • Implemented state transition detector
    • Basic relationship builder functionality
  2. Visualiser:

    • Deliver code to

      • generate a state transition and relationship graph of txn tree and account update forest. Attached is an example
      • summarise txn flow in ASCII and/or Markdown format. Attached is an example
      # Token Escrow Account Update Forest
      
      ## Account Types & Addresses
      
      ### Test Accounts
      - **Deployer** (Local Account 0)
        - Role: Deploys contracts and funds initial accounts
        - Type: Local blockchain test account
      
      - **Owner** (Local Account 1)
        - Role: Token admin operations and escrow control
        - Type: Local blockchain test account
      
      - **Alexa** (Local Account 2)
        - Role: Token holder and escrow depositor
        - Type: Local blockchain test account
      
      - **Billy** (Local Account 3)
        - Role: Token holder and escrow depositor 
        - Type: Local blockchain test account
      
      - **Jackie** (Local Account 4)
        - Role: Token recipient
        - Type: Local blockchain test account
      
      ### Contract Accounts
      - **Token Contract** (`PrivateKey.randomKeypair()`)
        - Type: FungibleToken
        - Features: 
          - Symbol: "abc"
          - Decimals: 9
          - Start State: Unpaused
      
      - **Escrow Contract** (`PrivateKey.randomKeypair()`)
        - Type: TokenEscrow
        - Features:
          - Owner: Local Account 1
          - Token Address: Token Contract's public key
      
      - **Admin Contract** (`PrivateKey.randomKeypair()`)
        - Type: FungibleTokenAdmin
        - Features:
          - Admin: Admin's public key
      
      ## Transaction Flow
      
      ### 1. Initial Setup & Deployment
      
      
      Deployer β†’ AdminContract
      β”‚  └─ Action: Fund (3 accounts)
      β”‚  └─ Fee: 1e8
      β”‚
      Deployer β†’ TokenContract
      β”‚  └─ Action: Deploy
      β”‚  └─ TokenId: {derived from token address}
      β”‚
      Admin β†’ TokenContract
         └─ Action: Initialize
         └─ Parameters: 
            - symbol: "abc"
            - decimals: 9
            - startPaused: false
      
      
      
      ### 2. Escrow Setup
      
      Deployer β†’ EscrowContract
      β”‚  └─ Action: Fund & Deploy
      β”‚  └─ Fee: 1e8
      β”‚
      EscrowContract β†’ TokenContract
         └─ Action: Approve Account Update
      
      
      ### 3. Token Distribution
      
      Owner β†’ Alexa
      β”‚  └─ Action: Mint 2e9 tokens
      β”‚  └─ Fee: 1e8
      β”‚
      Owner β†’ Billy
         └─ Action: Mint 3e9 tokens
         └─ Fee: 1e8
      
      
      ### 4. Escrow Operations
      
      Alexa β†’ EscrowContract
      β”‚  └─ Action: Deposit 2e9 tokens
      β”‚  └─ Fee: 1e8
      β”‚
      Billy β†’ EscrowContract
      β”‚  └─ Action: Deposit 3e9 tokens
      β”‚  └─ Fee: 1e8
      β”‚
      Owner β†’ EscrowContract
      β”‚  └─ Action: Withdraw 4e9 tokens
      β”‚  └─ Fee: 1e8
      β”‚
      EscrowContract β†’ Jackie
         └─ Action: Transfer 4e9 tokens
      
      
      ### 5. Failed Operation
      
      Jackie β•³ EscrowContract
         └─ Action: Direct withdraw attempt (1e9 tokens)
         └─ Status: REJECTED
         └─ Fee: 1e8
      
      
      ## Balance States
      
      ### Initial Token Distribution
      - Alexa: 2e9 tokens
      - Billy: 3e9 tokens
      - Escrow: 0 tokens
      - Jackie: 0 tokens
      
      ### After Deposits
      - Alexa: 0 tokens (all deposited)
      - Billy: 0 tokens (all deposited)
      - Escrow: 5e9 tokens
      - Jackie: 0 tokens
      
      ### After Withdrawal
      - Alexa: 0 tokens
      - Billy: 0 tokens
      - Escrow: 1e9 tokens
      - Jackie: 4e9 tokens
      
      
      
  • Below are code snippets indicating how the analyser and the visualiser could get invoked.

    Instantiate the classes

    const tracker = new AccountUpdateTrace();
    const visualizer = new ASCIITreeVisualizer();

    Trigger analyzer by taking a snaphot of the tree

    const deployTxnSnap = tracker.takeSnapshot(deployTxnAU, 'deploy')

    Invoke the visualizer like so:

    console.log(visualizer.visualizeChangeSummary(tracker.getSnapshots()));

    • Final Milestones (4 weeks):

      • Extend the solution to cater to the following

        • Usual, Archive Nodes
        • Blockberry APIs (experiment)
        • Mina Explorer (via Rest APIs)
      • NPM Package available in npm registry

      • Update relevant documentation


Project Timeline: 8 weeks

Week 1-2:

  • Develop State Analyser and Visualiser components
  • Build NPM package

Week 3-4:

  • Test Analyser, Visualiser
  • Add intricate details eg: approval data, errors
  • Explore setting up and querying usual, archive nodes and Blockberry APIs

Week 5-6:

  • Extend Analyser and Visualiser to nodes and apis
  • Build NPM package and push to NPM registry

Final 2 weeks:

  • Continue testing the code and npm package
  • Generate / Refine documentation

Budget Requested: 32,500 MINA

Budget Breakdown:

  1. Development Costs: 22,500 MINA

    • State Analyzer Development: 10,000 MINA
    • Visualiser Development: 12,500 MINA
  2. Testing and Quality Assurance: 5,000 MINA

  3. Documentation: 3,000 MINA

  4. Project Management: 2,000 MINA

Wallet Address:

B62qrF8xMKPPQMCwK2J7Ey47CGy1bKvaKhYNuYURN8qVohr1NJcnhRj

Team Info

Proposer Github:

Yash Bharti: YashBit (yash_b) Β· GitHub

Proposer Experience: If you have relevant experience building zkApps or other dApps, provide that information here. If available, include links.

  1. PSE - Ethereum Foundation Grantee
  2. Internet Computer Grantee
  3. Distributed Systems, Web Dev, ZK
  4. Rust, C++, Python, Next JS

Team Members:

  1. Krishanu Dhar: ronykris (krish) Β· GitHub
  2. Rute Figueiredo: rutefig (Rute Figueiredo) Β· GitHub

Risks & Mitigations

  1. Technical Complexity: Enhanced traversal and parsing of AccountUpdate forest may be more complex than anticipated.

    Mitigation: Begin with thorough analysis of existing code. Prioritize core functionalities and prototype early.

  2. Performance Issues: Increased detail in visualization may cause performance bottlenecks.

    Mitigation: Implement progressive rendering and lazy loading. Conduct regular performance testing.

  3. Compatibility with Future Mina Updates: Changes to Mina’s transaction structure could break the visualizer.

    Mitigation: Design a modular core engine. Maintain communication with Mina core team. Implement extensive unit testing.

  4. Scope Creep: Risk of exceeding the 2-month timeline due to feature expansion.

    Mitigation: Clearly define and prioritize features upfront. Use agile methodologies. Be prepared to adjust scope to meet timeline.

By anticipating these risks and implementing mitigations, we aim to ensure smooth development and delivery of the Enhanced Mina Transaction Visualizer within the proposed timeline and budget.

2 Likes

This proposal is funded.Account updates are difficult to understand for developers working on Mina. This project is aimed at developers to improve their experience of building on Mina

2 Likes