"The officer was in his car when he was shot by the militants carrying machine guns," said the official, adding that the attack is suspected to be linked with al-Qaeda.
Al Qaeda in the Arabian Peninsula (AQAP) has carried out several hit-and-run attacks since the Yemeni army drove it from its southern strongholds in Abyan and Shabwa provinces last month.
The dead man, Colonel Nasser al-Issai, was the latest in a string of army and intelligence officers to have been killed in Yemen. So far more than 22 other senior officers have died this year.
The West is concerned AQAP could use Yemen, which borders major oil producer Saudi Arabia, as a base for international attacks.
Apart from the fight against al Qaeda, the government faces a push by southern separatists for independence and battles with rebels from the Shi'ite Muslim Houthi movement, which is trying to extend its control over the north.