Tracking of human beings represents a hot research topic in the field of video analysis. It is attracting an increasing attention among researchers thanks to its possible application in many challenging tasks. Among these, action recognition, human/human and human/computer interaction require bodypart tracking. Most of the existing techniques in literature are model-based approaches, so despite their effectiveness, they are often unfit for the specific requirements of a body-part tracker. In this case it is very hard if not impossible to define a formal model of the target. This paper proposes a multi-anchor tracking system, which works on 8 bits color images and exploits the mutual information to track human body parts (head, hands, …) without performing any foreground/background segmentation. The proposed method has been designed as a component of a more general system aimed at human interaction analysis. It has been tested on a wide set of color video sequences and the very promising results show its high potential.