ThirdEye Infrastructure Requirements

​

ThirdEye components

• Coordinator: This is the API Server. It also powers root cause analysis (RCA) pipelines and ad hoc charting.
• Scheduler: Runs all the cron jobs and creates tasks
• GUI: Serves the React frontend for ThirdEye
• Worker: Executes the tasks
• MySQL: Provides state/metadata persistence for ThirdEye and serves the task queue

• Coordinator and Worker pods do most of the heavy lifting
• ThirdEye uses MySQL for task processing as well as for processing and storing anomalies, which can require significant DB resources
• Memory Consumption
  • The task that demands the most memory is the detection pipeline as it will fetch large chunks of data to process in-memory
  • The highest memory consumption will occur in cases where:
    • The granularity of alerts is low. For example, a process with a granularity of one minute will process significantly more data points than one with a granularity of one hour.
    • A coordinator previews an alert on a large timeframe
    • At the moment when an alert is created, because in the worker the alert is processed across all of the historical data.
• CPU Requirements
  • ThirdEye benefits greatly from parallel processing across multiple processors
  • You can configure the number of threads and workers available
  • More CPU greatly helps in executing multiple concurrent tasks and responding to API calls
• The ThirdEye backend can be run entirely in a single virtual machine. This means that the coordinator, scheduler, and worker can run in a single JVM process

​

Example deployments

​

Minimal deployment for development

​

Production examples

​

Standard setup

​

Enterprise setup

​

External resources

• AWS Instance Types
• GCP
  • GCP Instance Types
  • E2 Machine vs C3 Machine types. (Note: E2 Machines are less expensive.)