Skip to content

Performance Bottleneck: Optimize Data Aggregation in getAllData #8

@khdxsohee

Description

@khdxsohee

🐌 Current Performance Issue

The appscript function getAllData(targetMonth) currently processes data by:

  1. Fetching all payment data from the specific month's sheet (paidStudents set).
  2. Iterating through all students in the Student Registration sheet.
  3. For each student, checking their status, admission date, withdrawal date, and finally checking if they exist in the paidStudents set.

While functional, this approach uses a double-loop structure (an iteration over sheets and an iteration over students) which is inefficient and slow as the number of months and students increases. It may hit Google App Script's execution time limits in large deployments.

🎯 Proposed Optimization Goal

Our goal is to significantly reduce the execution time of the getAllData function, especially for large datasets (e.g., $1000+$ students and $2+$ years of data).

We should investigate methods to minimize the number of calls to SpreadsheetApp APIs and optimize data manipulation.

🔧 Possible Refactoring Strategies

We are looking for contributions on the best way to refactor this logic. Some potential strategies include:

  1. Reduce API Calls: Can we use a single SPREADSHEET.getRange().getValues() call to fetch all necessary student data instead of repetitive calls?
  2. Google Sheet Queries: Utilize Google Sheet Query Language (QUERY function) directly via SpreadsheetApp to fetch the registered students who are not yet paid for the month, allowing the spreadsheet engine to handle the heavy lifting.
  3. Caching: Implement a caching mechanism (e.g., CacheService) for static data like the list of registered students to avoid fetching it on every dashboard load.

Any contribution that improves the execution speed and efficiency of data aggregation is welcome!

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions