Technologies are provided for detecting anomalous data at runtime in an autonomous vehicle component that is indicative of a byzantine fault, and providing real-time remediation. Input and output data streams of the vehicle component can be analyzed to generate a normal operational model for the vehicle component. Using the model, deviations from a known steady state of operation of the vehicle component can be detected and flagged as potential faults. The vehicle component can then be isolated and/or restarted. Additionally or alternatively, a specification can be defined that specifies allowed component interactions in the autonomous vehicle system. The specification can be used to generate tests that can be used to validate the functional correctness of the vehicle components.