Researcher: Enemies at Mac's Gatekeeper have way around OS X security measure

Until fairly recently, Apple's latest Mac operating system was plagued with trojans – malware disguised as programs that users were willing to download from the Internet.

Between 2006 and 2011, researchers found more than a dozen significant variants in the wild. That era ended when Apple launched Gatekeeper in 2012, an antimalware feature for OS 10.7.3 and higher. Gatekeeper seemed to stem the tide. But new research from the security firm Synack shows that it’s relatively simple to sidestep Gatekeeper.

Patrick Wardle, director of research at Synack, says he can avoid Gatekeeper in software that incorporates external executable components, including libraries and extensions. Gatekeeper is designed to assure the authenticity of the main directory of software in the larger software package that Mac users download from the Web – but does nothing to check the authenticity of executables outside that main directory. Mr. Wardle says an attacker can hide malware in a library or extension that an authenticated program will launch with other add-on components.

"Once gatekeeper verifies an executable, it trusts it to launch other executables," says Wardle.

Software such as from Photoshop or Web browsers have been known to open external, often third-party plug-ins to expand their own capabilities.

Wardle is quick to say that this is not a vulnerability in Gatekeeper. Rather, checking executable programs bundled with authentic software is a useful feature not included in Gatekeeper. He says that Gatekeeper is great at doing what it was designed to do: authenticating that programs downloaded from the Internet match Apple certified versions of software. If an attacker changes any of the code, Gatekeeper can halt the infected program. What it is not designed to do is check that unsigned components included with the certified software are actually supposed to be there.

Wardle notified Apple over the summer of his method to circumvent Gatekeeper, and Apple is now working with him to develop both a short-term mitigation and a long-term patch for the issue. In the meantime, Wardle will present his research at Prague’s VirusBulletin conference on Oct. 1.

Apple did not reply to requests for comment.

If it seems like Wardle’s attack is so simple it should have come up in the past, Wardle agrees. "Known Mac malware is not sophisticated," says Wardle, who gave a talk at this year’s Black Hat hacker conference decrying the lack of creativity shown by Mac attackers.

Though Wardle's research now shows it can be beaten, it's hard to overestimate the impact Gatekeeper has had on Mac security as both a deterrent and a shield, says Ryan Naraine, Kaspersky Labs US director of its global research team.

"Gatekeeper became one of the most significant efforts to block Mac malware," he said. "It was transformative."

Wardle says consumers can increase the safety of downloaded software by ensuring they use HTTPS enabled sites when downloading software from the Web. This would prevent intermediaries from being able to insert malware.

"And if they don’t have an HTTPS option," he says, "call them and ask why."