The name originates from the Arab word for castle, 'Qumarix'. In Nazari times it was known as Hins Qumarix, or 'High Castle'. Other sources spell the name as Comarix, Qumaris or Qumarich.

The first fortress was built by the Romans, but the history of the town is pre-Roman and the main traces of occupation are Arabic.

When the Moors invaded Comares they made full use of its excellent location and developed the town into one of Axarquia's principle defensive strongholds. During the 11th century Comares became the regional administrative headquarters of Cutar, El Borge, Almachar and Moclinejo. By the 15th century, however, Moorish rule in Spain was declining and when Velez-Málaga fell to the Christians in May 1487. The last Moslem governor of Comares, Mohammed El Jabis, paid homage to the Catholic monarchs in the hope of staving off invasion. But in late 1487 it fell to the Christians.