Program monitoring based on automaton simulation

To check whether a program behaves in expectation, program monitoring systems are used for intrusion detection. This article presents a program monitoring system using automaton simulation based on the state graphs extracted from C programs through static analysis. For complete state graph construction, a pointer alias analysis method is proposed to solve the function pointers for obtaining actual control flows. After compiling, programs are instrumented with probes to report the internal states when they are running. A program monitor is built in the kernel of Linux system, which monitors the states of programs from probes and checks the paths of execution. This monitoring system could respond to the abnormal behaviors immediately to protect the systems and programs from further damages.