This paper describes a real time vision system that allows us to localize faces in video sequences and verify their identity. These processes are image processing techniques and the radial basis function (RBF) neural network approach. The robustness of this system has been evaluated quantitatively on eight video sequences. We have adapted our model for an application of face recognition using the Olivetti Research Laboratory (ORL), Cambridge, U.K., database so as to compare performance against other systems. We also describe three hardware implementations of our model on embedded systems based, respectively, on field programmable gate array (FPGA), zero instruction set computer (ZISC) chips, and digital signal processor (DSP) TMS320C62. We analyze the algorithm complexity and present results of hardware implementations in terms of resources used and processing speed. The success rates of face tracking and identity verification are, respectively, 92% (FPGA), 85% (ZISC), and 98.2% (DSP). For the three embedded systems processing speeds for images size of 288 x 352 are, respectively, 14 images/s, 25 images/s, and 4.8 images/s.