Graves Concerns

In the security profession, we have a maxim that security is not a product. It’s a reminder that security doesn’t result from plugging in devices, but through continuous integration of security into design, development, management, and operations. I’d add another maxim: security is not a checklist.

When I was in QSA training a few years back, our trainer claimed that no one who was PCI DSS compliant had ever suffered a data breach. He hedged this bold statement by suggesting that anyone who had been certified as PCI DSS compliant and later suffered a breach must have fallen out of compliance by the time the breach happened. It was an entertaining exercise in circular logic: PCI DSS prevents security breaches, so obviously anyone who suffered a security breach couldn’t have been PCI DSS compliant.

Well, Heartland Payment Systems, who may have suffered the largest breach in history (giving executives at TJX something to celebrate), was certified as PCI DSS compliant. That suggests at least three possibilities:

1. Heartland was PCI DSS compliant when they were audited, but fell out of compliance by the time of they were breached;
2. Heartland wasn’t PCI DSS compliant, but their QSA said they were; or
3. PCI DSS doesn’t actually prevent compliant organizations from suffering a breach.

Each of the first two conclusions would be reasonable. A PCI DSS assessment is a snapshot in time, and business are constantly changing. And because they are paid by the companies they assess, it’s fair to wonder whether QSAs are truly independent. The third conclusion is more than reasonable, it’s certain: PCI DSS compliance doesn’t guarantee security. That should be obvious. But maybe it’s not.

PCI’s strength and weakness is that it’s a checklist of detailed requirements. Its specificity is an improvement over laws like HIPAA, which calls for protecting against “reasonably anticipated threats” while considering the size of the organization and the costs of the security measures. It’s a flexible approach, but it doesn’t provide many answers. Are firewalls required? Does internal traffic have to be encrypted? It depends.

As a checklist, PCI DSS is more to the point. Companies know exactly what’s expected. They have to have firewalls between untrusted networks and any cardholder data environment (PCI DSS Requirement 1.2), install personal firewall software on laptops (Requirement 1.4), use anti-virus software (Requirement 5.1), and so on. There’s very little “it depends” in the PCI DSS requirements.

But companies sometimes think the checklist is all they need—that once they’ve checked “compliant” next to all the requirements, they’re done (until the next audit rolls around). They fall into the trap of thinking that a checklist item intended to mandate a minimum level of adequate security is also the most they need to do. They forget that being able to answer “yes, we have a process” to a checklist item is not as important as whether that process works. Then, when data is lost, they point to the checklist and ask what more they were supposed to do. That’s when a reasonableness standard starts looking awfully good.

The checklist is necessary, because there’s too much wiggle room and too much ambiguity without one. But just as security is not a product, it is also not a checklist. It is, as always, a process—one that a checklist can inform, and sometimes measure, but never complete.