In this recipe, we’ll use Startree ThirdEye to monitor metrics critical to rideshare customer experience, such as cancellations and wait time. In addition to monitoring, we’ll also perform a root cause analysis within ThirdEye to identify dimensions that have changed and contributed the most to the outlier/deviation in the metric (spike or drop or trend change).

​
Challenges faced by rideshare apps

In order to be competitive, rideshare apps must provide fast and timely rides for their customers. When wait times and ride cancellations begin to spike, quick action is critical to restoring customer satisfaction. Wait times can be affected by external factors such as weather, traffic, driver availability, and more. When wait times are too long, customers begin to look for alternatives. This often leads to customers cancelling their current ride in order to try for a closer driver or choosing an alternative option. Cancelled rides add extra stress and costs to a system already struggling to provide timely rides. Quickly and confidently responding to changes in wait time and other key metrics will have a big impact on customer satisfaction and retention.

​
How anomaly detection helps

In order to respond to deviations in key metrics, we need a system that detects these deviations and notifies us that they have occurred. When the deviation is large enough, the side effects are readily apparent; site outages, angry customers, and damage to the brand. ThirdEye uses statistical models and dimensional analysis to detect smaller deviations and anomalies before they become big problems. Aggregate timeseries can often mask anomalies that occur in a single dimension of the data. For example, a single city might see wait times increase 10x, but this increase may have a negligible effect on the aggregate wait time across all cities. ThirdEye can run anomaly detection across multiple dimensions of data to detect these local anomalies. ThirdEye automatically detects when our key metrics unexpectedly deviate from their normal behavior, even across multiple dimensions. ThirdEye can report these anomalies to Slack, email, or an external webhook. Anomaly detect for ride cancellations: Anomaly detection for ride cancellations

​
Interested in trying out StarTree ThirdEye?

Follow this link to find different deployment options for trying out StarTree ThirdEye.

​
Architecture diagram

image

​
Steps to try StarTree ThirdEye anomaly detection for real-time or near real-time monitoring

  • Data preparation and availability
  • Data ingestion
  • Alert creation and notifications
  • Tuning anomaly detection models (if needed based on use-case and context)
  • Anomaly reporting
  • Root cause analysis/anomaly investigation

​
Data preparation and availability

  • Identify key metrics to monitor.
    This recipe uses:
    • Ride_Cancellations
    • Ride_Request_WaitTime
    • Driver_Request_API_Errors
  • Identify the dimensions that are applicable for a given metric.
    This recipe uses:
    • City
    • Device_Type
    • RideType
  • Decide granularity for your detection:
    • Granularity enables you for real-time or near real-time monitoring (hourly/daily/weekly time series). The sample data is aggregated into hourly buckets, so this recipe uses hourly granularity.
Types of time series data image

​
Data ingestion

  1. Download this CSV file link.
  2. Ingest this CSV data to Pinot using the Pinot API.
Here’s a preview of the first 5 rows:
timestampRideTypeCityDevice_TypeOS_VersionRide_CancellationsRides_CompletedRide_Request_WaitTimeDriver_Request_API_Errors
1512867600SharedLos_AngelesMobileAndroid6241.517411320.58332434.66321772.2415
1512871200SharedLos_AngelesMobileAndroid6840.427211429.29842611.57661886.1431
1512874800SharedLos_AngelesMobileAndroid7421.947411509.16252859.70991970.7529
1512878400SharedLos_AngelesMobileAndroid7780.71511350.57482986.99792076.0819
1512882000SharedLos_AngelesMobileAndroid7996.776211804.10362976.74762238.1518
The Pinot schema looks like this: 
{
  "schemaName": "rideshare",
  "dimensionFieldSpecs": [
    {
      "name": "ver",
      "dataType": "STRING"
    },
    {
      "name": "RideType",
      "dataType": "STRING"
    },
    {
      "name": "City",
      "dataType": "STRING"
    },
    {
      "name": "Device_Type",
      "dataType": "STRING"
    },
    {
      "name": "OS_Version",
      "dataType": "STRING"
    }
  ],
  "metricFieldSpecs": [
    {
      "name": "Ride_Cancellations",
      "dataType": "FLOAT"
    },
    {
      "name": "Rides_Completed",
      "dataType": "FLOAT"
    },
    {
      "name": "Ride_Request_WaitTime",
      "dataType": "FLOAT"
    },
    {
      "name": "Driver_Request_API_Errors",
      "dataType": "FLOAT"
    }
  ],
  "dateTimeFieldSpecs": [
    {
      "name": "timestamp",
      "dataType": "LONG",
      "format": "1:SECONDS:EPOCH",
      "granularity": "1:SECONDS"
    }
  ]
}

​
Alert creation and notifications

To create our alerts, we need to inspect the timeseries for trends, seasonality, and obvious outliers. ThirdEye’s alert preview lets us view the timeseries and the predicted anomalies. To start, we’ll preview the data using the simplest alert temlate, startree-threshold. Copy and paste the following into ThirdEye’s alert json editor: 
{
  "name": "",
  "description": "",
  "cron": "0 0 5 ? * MON-FRI *",
  "template": {
    "name": "startree-threshold"
  },
  "templateProperties": {
    "dataSource": "mypinot",
    "dataset": "rideshare",
    "aggregationColumn": "Ride_Request_WaitTime",
    "aggregationFunction": "SUM",
    "seasonalityPeriod": "P7D",
    "lookback": "P90D",
    "monitoringGranularity": "PT1H",
    "sensitivity": "3",
    "max": "100000",
    "min": "0"
  }
}
You should now see this timeseries in the preview: Timeseries of alert wait time From the preview, we see a fairly cyclic pattern each day, with some variation between the days, and a large outlier towards the end of March. The cyclic pattern is called the seasonality. In fact, all our key metrics, wait time, cancellations, and api errors, show a seasonality period of 1 day. At the beginning of the day, the metrics are fairly low, they reach a peak towards the middle of the day, and end at roughly the same value as the start of the day. A simple threshold alert will not work for this data, since the threshold would depend on the time of day. We also want to monitor different dimensions of the data, which would require their own time-dependent thresholds. Instead, we’ll use the startree-ets-dx template which is designed to model data with seasonality and trends.

​
Alert Configuration for Wait Time

{
  "id": 36736,
  "name": "Ride_Request_WaitTime",
  "description": "",
  "template": {
    "name": "startree-ets-dx"
  },
  "templateProperties": {
    "dataSource": "mypinot",
    "dataset": "rideshare",
    "timeColumn": "timestamp",
    "timeColumnFormat": "EPOCH",
    "aggregationFunction": "SUM",
    "seasonalityPeriod": "P1D",
    "lookback": "P28D",
    "monitoringGranularity": "PT1H",
    "aggregationColumn": "Ride_Request_WaitTime",
    "queryFilters": "${queryFilters}",
    "enumerationItems": [
      {
        "name": "overall",
        "params": {
          "queryFilters": ""
        }
      },
      {
        "params": {
          "queryFilters": " AND City = 'Los_Angeles' and RideType = 'Standard'"
        }
      },
      {
        "params": {
          "queryFilters": " AND City = 'Los_Angeles' and RideType = 'Shared'"
        }
      },
      {
        "params": {
          "queryFilters": " AND City = 'Los_Angeles' and RideType = 'Premium'"
        }
      },
      {
        "params": {
          "queryFilters": " AND City = 'New_York' and RideType = 'Standard'"
        }
      },
      {
        "params": {
          "queryFilters": " AND City = 'New_York' and RideType = 'Shared'"
        }
      },
      {
        "params": {
          "queryFilters": " AND City = 'New_York' and RideType = 'Premium'"
        }
      },
      {
        "params": {
          "queryFilters": " AND City = 'San_Francisco' and RideType = 'Standard'"
        }
      },
      {
        "params": {
          "queryFilters": " AND City = 'San_Francisco' and RideType = 'Shared'"
        }
      }
    ],
    "pattern": "UP",
    "sensitivity": "3"
  },
  "cron": "0 0 5 ? * MON-FRI *",
  "lastTimestamp": 1677690000000,
  "active": true,
  "created": 1677686514149,
  "updated": 1677708931249,
  "owner": {
    "principal": "no-auth-user"
  }
}

​
Alert Configuration for Cancellations

{
  "id": 36735,
  "name": "Ride_Cancellations",
  "description": "",
  "template": {
    "name": "startree-ets-dx"
  },
  "templateProperties": {
    "dataSource": "mypinot",
    "dataset": "rideshare",
    "timeColumn": "timestamp",
    "timeColumnFormat": "EPOCH",
    "aggregationFunction": "SUM",
    "seasonalityPeriod": "P1D",
    "lookback": "P28D",
    "monitoringGranularity": "PT1H",
    "sensitivity": "3",
    "aggregationColumn": "Ride_Cancellations",
    "queryFilters": "${queryFilters}",
    "enumerationItems": [
      {
        "name": "overall",
        "params": {
          "queryFilters": ""
        }
      },
      {
        "params": {
          "queryFilters": " AND City = 'Los_Angeles' and RideType = 'Standard'"
        }
      },
      {
        "params": {
          "queryFilters": " AND City = 'Los_Angeles' and RideType = 'Shared'"
        }
      },
      {
        "params": {
          "queryFilters": " AND City = 'Los_Angeles' and RideType = 'Premium'"
        }
      },
      {
        "params": {
          "queryFilters": " AND City = 'New_York' and RideType = 'Standard'"
        }
      },
      {
        "params": {
          "queryFilters": " AND City = 'New_York' and RideType = 'Shared'"
        }
      },
      {
        "params": {
          "queryFilters": " AND City = 'New_York' and RideType = 'Premium'"
        }
      },
      {
        "params": {
          "queryFilters": " AND City = 'San_Francisco' and RideType = 'Standard'"
        }
      },
      {
        "params": {
          "queryFilters": " AND City = 'San_Francisco' and RideType = 'Shared'"
        }
      }
    ],
    "impactThreshold": "1000",
    "pattern": "UP"
  },
  "cron": "0 0 5 ? * MON-FRI *",
  "lastTimestamp": 1677693600000,
  "active": true,
  "created": 1677686514097,
  "updated": 1677695955731,
  "owner": {
    "principal": "no-auth-user"
  }
}

​
Alert Configuration for Api Errors

{
  "name": "Driver_Request_API_Errors",
  "description": "",
  "template": {
    "name": "startree-ets-dx"
  },
  "templateProperties": {
    "dataSource": "mypinot",
    "dataset": "rideshare",
    "timeColumn": "timestamp",
    "timeColumnFormat": "EPOCH",
    "aggregationFunction": "SUM",
    "seasonalityPeriod": "P7D",
    "lookback": "P28D",
    "monitoringGranularity": "PT1H",
    "sensitivity": "3",
    "aggregationColumn": "Driver_Request_API_Errors",
    "queryFilters": "${queryFilters}",
    "enumerationItems": [
      {
        "name": "overall",
        "params": {
          "queryFilters": ""
        }
      },
      {
        "params": {
          "queryFilters": " AND City = 'Los_Angeles' and RideType = 'Standard'"
        }
      },
      {
        "params": {
          "queryFilters": " AND City = 'Los_Angeles' and RideType = 'Shared'"
        }
      },
      {
        "params": {
          "queryFilters": " AND City = 'Los_Angeles' and RideType = 'Premium'"
        }
      },
      {
        "params": {
          "queryFilters": " AND City = 'New_York' and RideType = 'Standard'"
        }
      },
      {
        "params": {
          "queryFilters": " AND City = 'New_York' and RideType = 'Shared'"
        }
      },
      {
        "params": {
          "queryFilters": " AND City = 'New_York' and RideType = 'Premium'"
        }
      },
      {
        "params": {
          "queryFilters": " AND City = 'San_Francisco' and RideType = 'Standard'"
        }
      },
      {
        "params": {
          "queryFilters": " AND City = 'San_Francisco' and RideType = 'Shared'"
        }
      }
    ],
    "pattern": "UP",
    "impactThreshold": "100"
  },
  "cron": "0 0 5 ? * MON-FRI *"
}

​
Anomaly reporting

Now that the alerts are created, we view the detected anomalies for each alert and it’s dimensions. To see anomalies for a single alert, click Alerts in the sidebar, then click on the alert you want to inspect. To see anomalies for all alerts, click on Anomalies in the sidebar. Below are screenshots from the anomaly detection results. The graphs are zoomed in to show the most interesting results. Anomalies in wait time Anomalies in wait time Anomalies in ride cacellations Anomalies in ride cancellations Anomalies in API errors Anomalies in API errors ThirdEye has several options for reporting anomalies:

​
Root-cause analysis

Follow this guide to perform root cause analysis with heatmaps, custom events, and other signals.