PMM-14554 RTA in-memory store #4890
Open
+533
−7
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Implement database schema for feature configuration using Option 3: - Create new agent record type for real-time analytics - Add MongoDBRealtimeAgentType constant to agent types - Use existing agents table infrastructure - Associate agent with service - Agent status (running/stopped) represents enabled/disabled - Retrieve credentials from existing MongoDB agents during creation Changes: - Add MongoDBRealtimeAgentType constant to models - Update DSN helpers to handle MongoDB realtime agent - Add CreateMongoDBRealtimeAgent helper function - Update proto file with new agent type and messages - Add agent to Files() method for certificate handling No database migration needed as existing agents table supports new type.
- Introduced RTAOptions type for future Real-Time Analytics configuration. - Implemented database/sql/driver.Valuer and Scanner interfaces for RTAOptions. - Updated Agent and CreateAgentParams to include RTAOptions. - Modified database schema to add rta_options column to agents table.
- Deleted MongoDBRealtimeAgent message and its associated parameters from the proto file. - Removed AGENT_TYPE_MONGODB_REALTIME_AGENT from AgentType enum. - Cleaned up related comments and documentation for clarity.
- Integrated RTAOptions into the Agent model, including updates to the Columns, String, Values, and Pointers methods to accommodate the new field. - Adjusted the database schema to include the rta_options column for proper data handling.
…eation logic. Upgraded actions/checkout to a specific commit and actions/setup-node to a specific version. Added logic to prevent duplicate MongoDB Realtime Agents by checking for existing agents before creation.
…-14567 # Conflicts: # managed/models/agent_helpers.go # managed/models/agent_model_reform.go # managed/models/database.go
…ate string representation length
- Introduced the RealtimeAnalyticsService with methods to list currently running RTA agents. - Added necessary proto files for gRPC and RESTful API integration. - Implemented validation logic for request and response messages. - Created service logic to handle agent data retrieval and filtering by cluster. - Added unit tests to ensure functionality and correctness of the service methods.
- Updated PMM Server and MongoDB images in GitHub Actions workflows to specific versions. - Enhanced the Realtime Analytics service by modifying the ListRunningRealtimeAgents method signature for clarity. - Changed test assertions to use require.Empty for better readability.
- Changed the initialization of the Agents slice in the ListRunningRealtimeAgents method from using make to a shorthand slice literal for improved readability.
- Introduced ChangeRealtimeAnalyticsRequest and ChangeRealtimeAnalyticsResponse messages for enabling or disabling Real-Time Analytics (RTA) for specific services or clusters. - Implemented ChangeRealtimeAnalytics method in RealtimeAnalyticsService to handle RTA state changes. - Added validation logic to ensure either service_id or cluster is specified in the request. - Updated gRPC and HTTP handlers to support the new functionality. - Enhanced unit tests to cover various scenarios for enabling and disabling RTA.
- Updated assertions in TestChangeRealtimeAnalytics to use require.Empty for improved readability and clarity in test outcomes.
…e files for consistency and clarity.
…AgentType in service and test files for consistency.
Co-authored-by: Alex Demidoff <alexander.demidoff@percona.com>
…DBAgentType in service and test files for consistency.
- Introduced a new in-memory store for real-time analytics queries to enhance performance and reduce lock contention. - Updated the RTA service to utilize the new store, allowing for concurrent operations. - Modified tests to accommodate the new store structure and ensure functionality. - Added new entries to .gitignore for generated files.
… and simplify ChangeRealtimeAnalytics request. Update documentation to reflect that RTA can only be enabled/disabled for a service. Remove unused code and validation for cluster in related requests and tests.
…eting and enforce service ID requirement
…-14574-enable-disable-realtime-agent # Conflicts: # api/realtimeanalytics/v1/realtimeanalytics.pb.go # api/realtimeanalytics/v1/realtimeanalytics.pb.gw.go # api/realtimeanalytics/v1/realtimeanalytics.pb.validate.go # api/realtimeanalytics/v1/realtimeanalytics.proto # api/realtimeanalytics/v1/realtimeanalytics_grpc.pb.go # managed/models/agent_helpers.go # managed/services/realtimeanalytics/service.go # managed/services/realtimeanalytics/service_test.go
…RTA configuration
…data management. Updated methods to handle query buckets with TTL, ensuring expired buckets are cleaned up correctly. Enhanced tests to validate bucket expiration and cleanup functionality.
…luster parameter. Updated related tests to reflect the change, ensuring consistency and clarity in query retrieval.
…ealtime Analytics Store tests, enhancing accuracy in shard usage verification.
ademidoff
requested review from
JiriCtvrtka and
idoqo
and removed request for
a team
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PMM-14554
Link to the Feature Build: SUBMODULES-0
If this PR adds, removes or alters one or more API endpoints, please review and add or update the relevant API documentation as well:
If this PR is related to some other PRs in this or other repositories, please provide links to those PRs: