@gentlementlegen @whilefoo rfc on kernel-plugin interface design, and enabling negative permits (at least for one plugin to subtract rewards from the others in an invocation)

Similar to how GitHub actions supports capturing output from each step in CI, we should do the same.

We already support that - you can use any output from a plugin as input to another plugin

Then the kernel can sum the requested permits and post them all in a single comment at the end when the issue is closed as complete.

I'm not sure if kernel should be responsible for dealing with permits, it should only call plugins. I don't understand why we need to sum permits, won't conversation-rewards generate all rewards and permit-generation will generate permits based on that?

Regarding comment output, if we support full HTML comment output from each plugin, we could generate the comment body by concatenating all of the comment outputs of every plugin invoked as a response to the given event.

That's a good idea, a standard property like comment and then the kernel posts a comment when all plugins finish executing. So conversation-rewards can output a comment about reward summary and permit-generation outputs permits, then the kernel posts 1 comment containing both.

So I guess for inputs every plugin should support some standard properties which right now aren't clear to me. I presume we will pass along event context from the kernel. I suppose we can pass in a string as the arbitrary input value, similar to a command line interface. This will allow us to serialize complex json objects if needed, or pass in simple string parameters to plugins if that's all they need?

For now all plugins have these standard inputs. If the need arises we can add more

I'm not sure if kernel should be responsible for dealing with permits, it should only call plugins. I don't understand why we need to sum permits, won't conversation-rewards generate all rewards and permit-generation will generate permits based on that?

That's a good idea, a standard property like comment and then the kernel posts a comment when all plugins finish executing. So conversation-rewards can output a comment about reward summary and permit-generation outputs permits, then the kernel posts 1 comment containing both.

The kernel should be responsible for dealing with permits in a similar way that I propose it would be dealing with the comments. I'll try my best to explain:

won't conversation-rewards generate all rewards

No, there are many future planned incentives that should not be handled by conversation-rewards for example, providing rewards for completing pull request reviews (i.e. approve, request changes) providing time estimates (Time: label.)

Imagine a future where organizations codify their own direct financial incentives for their contributors. Conversation rewards is definitely a cool general purpose example, but its far from the only idea that I have (and I'm sure other partners will have.) To accomodate this future, we need to support "payment requests" from every plugin invoked in a webhook handler event chain.

Plugins should also have the ability to modify the rewards outputs of others.

One planned example of this is our developer relations (referral) system, with "zero sum" rewards. This will subtract from a contributor's reward, and pay the person who referred them. You can read more about it in my proposal here.

/stop

/stop

/start

/start

/ask any inputs for solutions on this conversation?

Based on the comprehensive discussion in Issue #3, here are several input solutions to address the key challenges and streamline the project's workflow:

1. Introduce a Dedicated Permit Aggregator Module

Problem: The current approach where the kernel is responsible for collecting and summing permits from various plugins increases coupling between the kernel and plugins, making the system less modular and harder to maintain.

Solution:

• Create a Separate Permit Aggregator: Develop a standalone module specifically designed to handle the aggregation of permits from different plugins. This module will interface with the kernel and plugins through a well-defined API, reducing direct dependencies.
• Workflow Integration:
  1. Plugins Emit Permits: Each plugin generates its own paymentRequest and metadata without needing to know about other plugins.
  2. Aggregator Collects Permits: The Permit Aggregator listens for these permit requests and aggregates them.
  3. Single Permit Generation: Once all permits are collected, the aggregator sends a consolidated permit to the kernel for posting.

Benefits:

• Enhances modularity and scalability.
• Simplifies testing and maintenance by isolating permit logic.
• Facilitates easier addition of new plugins without modifying the kernel.

2. Standardize Plugin-Kernel Interface

Problem: Lack of a standardized interface leads to inconsistent data handling and makes it challenging to manage interactions between plugins and the kernel.

Solution:

• Define a Consistent Interface: Establish a TypeScript interface that all plugins must adhere to when communicating with the kernel. For example:

  interface PluginOutput {
    paymentRequest?: PaymentRequest;
    comment?: string;
    metadata?: Record<string, any>;
  }
  
  interface PluginInput {
    eventContext: EventContext;
    // Additional standardized properties
  }

• Enforce via TypeScript: Utilize TypeScript's type-checking capabilities to ensure all plugins implement the standardized interface, reducing runtime errors and enhancing reliability.

Benefits:

• Ensures consistency across all plugins.
• Simplifies the integration process for new plugins.
• Enhances type safety and developer experience.

3. Implement Zero-Sum and Non-Zero-Sum Incentive Models

Problem: Balancing DevRel incentives without adversely affecting developer rewards is challenging, especially when considering scalability and fairness.

Solution:

• Configurable Incentive Modes:
  • Zero-Sum Mode: A configurable percentage (e.g., 5%) of developer rewards is allocated to DevRel. This ensures that incentives are balanced without introducing additional costs.
  • Non-Zero-Sum Mode: DevRel receives additional rewards without reducing developer payouts. This can be beneficial during initial growth phases.
• Configuration Management:
  • Store incentive configurations in Supabase, allowing dynamic adjustments without redeploying the system.
  • Provide toggle switches and percentage configurations through environment variables managed by dotenv.

Implementation Example:

interface IncentiveConfig {
  mode: 'zero-sum' | 'non-zero-sum';
  percentage?: number; // Applicable for zero-sum mode
}

// Fetch configuration from Supabase
const config: IncentiveConfig = await fetchIncentiveConfig();

Benefits:

• Offers flexibility to switch between incentive models based on organizational needs.
• Ensures fair distribution of rewards.
• Facilitates experimentation and data-driven adjustments.

4. Enhance Security Against Sybil Attacks

Problem: The referral system is vulnerable to abuse through fake profiles, potentially skewing reward distributions.

Solution:

• Implement Validation Checks:
  • Profile Age: Only allow referrals from GitHub profiles older than a certain threshold (e.g., 6 months).
  • Contribution Activity: Require a minimum number of contributions in the past year to qualify.
  • Profile Completeness: Ensure profiles have linked social accounts (e.g., Twitter, LinkedIn) and filled-out information.
• Leverage Supabase for Data Management:
  • Store validated referral data securely.
  • Use Supabase's authentication and API capabilities to manage and verify referrals.

Benefits:

• Reduces the risk of fraudulent referrals.
• Ensures that rewards are distributed to genuine contributors.
• Maintains the integrity of the incentive system.

5. Integrate Flyway with Supabase for Schema Management

Problem: Managing database schema changes manually can lead to inconsistencies and technical debt over time.

Solution:

• Use Flyway for Migrations:
  • Implement Flyway in GitHub Actions to automate schema migrations.
  • Maintain version-controlled migration scripts to ensure consistency across development, staging, and production environments.
• Continuous Integration Setup:
  • Configure GitHub Actions workflows to apply Flyway migrations automatically upon merging changes to specific branches.

Implementation Example:

name: Database Migration

on:
  push:
    branches:
      - main

jobs:
  migrate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Set up Flyway
        run: |
          curl -L https://flywaydb.org/getflyway | tar xvz
          export PATH=$PATH:$(pwd)/flyway-*
      - name: Apply Migrations
        run: flyway migrate -url="jdbc:postgresql://<supabase-url>" -user="<user>" -password="<password>"

Benefits:

• Ensures database schemas remain consistent across environments.
• Simplifies the deployment process.
• Reduces the risk of human error during schema changes.

6. Develop Comprehensive Onboarding Documentation

Problem: New contributors struggle to understand the codebase structure and module interactions, hindering effective participation.

Solution:

• Create Detailed Documentation:
  • Code Structure Overview: Provide diagrams and explanations of the overall architecture, highlighting how plugins interact with the kernel and other modules.
  • Module Interaction Maps: Illustrate the flow of data between different plugins and the Permit Aggregator.
  • Contribution Guidelines: Outline steps for setting up the development environment, submitting pull requests, and best practices for coding standards.
• Utilize README Files and Wikis:
  • Maintain up-to-date README files in each repository.
  • Use GitHub Wikis to host comprehensive guides and tutorials.

Benefits:

• Accelerates the onboarding process for new contributors.
• Reduces the learning curve associated with understanding the system.
• Enhances overall project transparency and accessibility.

7. Automate Issue Specification Updates with LLM Integration

Problem: Keeping issue specifications aligned with ongoing discussions is time-consuming and prone to inconsistencies.

Solution:

• Develop a /rewrite Command Plugin:
  • Functionality: Parses the conversation within an issue and uses a Language Model (LLM) to generate a summarized and updated issue specification.
  • Workflow:
    1. Trigger: Activated via a /rewrite command or automatically upon detecting specific triggers (e.g., label changes).
    2. Processing: Collects relevant comments and discussions, feeds them into the LLM, and generates an updated specification.
    3. Output: Posts the updated specification as a new comment and/or updates the issue body with a link to the latest spec comment.
• Maintain Revision History: Ensure that all updates are tracked, allowing contributors to reference previous versions if needed.

Implementation Considerations:

• Validation: Have moderators review automated updates to ensure accuracy and relevance.
• Customization: Allow repository owners to define specific triggers and formatting preferences for the rewritten specifications.

Benefits:

• Maintains up-to-date and relevant issue specifications.
• Reduces manual effort in summarizing and updating issues.
• Enhances clarity and direction for contributors working on tasks.

8. Enforce Code Quality with Husky, ESLint, and Prettier

Problem: Inconsistent code formatting and quality can lead to maintainability issues and integration problems.

Solution:

• Set Up Pre-commit Hooks with Husky and lint-staged:
  • Husky: Automates the running of scripts at various Git hooks.
  • lint-staged: Runs linters on staged files before a commit is finalized.
• Configure ESLint and Prettier:
  • ESLint: Enforce coding standards and identify potential errors.
  • Prettier: Maintain consistent code formatting across the codebase.

Implementation Example:

// package.json
{
  "husky": {
    "hooks": {
      "pre-commit": "lint-staged"
    }
  },
  "lint-staged": {
    "*.{ts,js}": [
      "eslint --fix",
      "prettier --write"
    ],
    "*.{css,md}": [
      "prettier --write"
    ]
  }
}

Benefits:

• Ensures code consistency and quality across all commits.
• Prevents common errors and enforces best practices.
• Enhances developer productivity by automating formatting and linting tasks.

9. Leverage TypeScript Development Tools

Problem: Managing a TypeScript codebase requires efficient tooling to enhance development workflow and maintain code integrity.

Solution:

• Use ts-node for Development: Allows running TypeScript files directly without precompiling, speeding up the development process.
• Utilize tsup for Bundling: Simplifies the bundling process for TypeScript projects, ensuring optimized builds for production.
• Integrate Type Definitions: Ensure all modules and plugins have proper TypeScript definitions to leverage type safety and IntelliSense features.

Benefits:

• Streamlines the development workflow.
• Enhances code reliability through type safety.
• Simplifies the build process, ensuring efficient and optimized production artifacts.

10. Establish a Feedback Loop for Continuous Improvement

Problem: Without structured feedback, it's challenging to identify areas for improvement and measure the effectiveness of implemented solutions.

Solution:

• Create Dedicated Communication Channels:
  • Telegram Group for DevRels: Facilitate real-time discussions, feedback, and updates.
  • Regular Surveys: Collect feedback from both developers and DevRel representatives to assess the incentive system and module functionalities.
• Implement Monitoring and Analytics:
  • Supabase Integration: Track key metrics related to permit generation, reward distributions, and system performance.
  • Dashboard Setup: Provide visual insights into system operations, enabling data-driven decision-making.

Benefits:

• Encourages active participation and continuous feedback from all stakeholders.
• Identifies pain points and areas for enhancement promptly.
• Supports iterative improvements, aligning the system with user needs and organizational goals.

By implementing these solutions, the project can achieve greater modularity, maintainability, and scalability while ensuring a fair and efficient incentive system for both developers and DevRel representatives. These inputs aim to address the core challenges discussed and provide a clear path forward for the team's ongoing efforts.