Query Generation Across Multiple Data St...

In this talk, we’ll discuss how we define and query cubes across multiple data stores for reporting purposes. With a single definition, we are able to decide at query time the best table/data source to answer a given request. We must take into consideration things such as time zone conversion, data availability, supported fact/dim based operations, request granularity, defined constraints, time range of request, and etc. Ultimately, our request is answered using Hive or RDBMS or Druid. This allows us to take advantage of performance characteristics of each data store while also allowing for a single interface for querying. Our goal isn’t to create a unified SQL layer which can be used to query multiple data stores. Our goal is to define a single view of the data where we can define post aggregates or other derived expressions which can later be used to programmatically generate a query for the target data store.