Efficient extraction of provenance information from database query execution logs
Systems, methods, and devices are described for efficient extraction of provenance information from database query execution logs. A set of event records extracted from one or more database query execution logs is received. Groups of related event records are identified, and each defined as an activity. For each activity, an activity data structure is generated based on the associated event records. The data structure encodes query execution dependencies for the activity. For each activity, runtime information is extracted by identifying and instantiating process entities and relationships based on the associated event records and activity data structure. For each activity, lineage information is extracted by identifying and instantiating dataset entities and their lineage relationships based on query text within the associated event records. The lineage information is mapped to the runtime information for each activity to generate and store a provenance data model for use by one or more provenance applications.