Security is an interesting dimension of software quality; essentially invisible to users, except for the inconvenience of managing passwords, public keys, and the like. Like performance, it is also difficult to write security requirements.
Security came up a fair amount at this year’s Philly ETE. Unfortunately I only made it to one of the talks that was directly on security issues, but I was able to attend a talk on handling security with third party libraries, by Kymberlee Price.
Third party libraries encompass a wide range of things – linked libraries, parts of the operating system, Java jar files, and things you get from NPM. The security industry has a variety of tools for scanning for issues, but if you’re managing a software project, third party libraries are more likely to be ignored, harder to track, and are likely to get new releases at inconvenient times.
One thing I found interesting was that several iPhone vulnerabilities where in a font library (FreeType)1. OpenSSL is also well-known as a popular target now, as well as a PNG library2. One of the more recent developments in the industry is higher quality marketing materials for exploits, as finding an exploit in a popular library is presumed to be a big career boost.
Known exploits are more difficult to track than the average software defect, as each vendor or project maintainer uses different bug tracking systems and disclosure policies. When defects are mentioned in the news, the associated statistics tend to be unclear. E.g., one could count individual defects requiring a code commit or the deployment size of a defect (# of models of phones with it, # of devices with it). In larger applications, it’s also possible to depend on multiple copies/versions of the same library.
When these are disclosed to a central US government database, and they are assigned a CVE number. If there isn’t a disclosure, they wouldn’t get a number. Some vendors fix these silently, and there are likely many issues only known in the blackhat community. There is a secondary market for vulnerabilities, so one of the criteria for evaluating how quickly you want to upgrade a broken library is the existence of known exploits (public or for sale).
These issues are apparently rated on a 10 point scale – 10 being your server is compromised, 5 being data loss. If data loss actually affected a password in your system, from a user’s perspective this is effectively as bad as a 10.
Another interesting challenge is that libraries vary in their release schedule (yearly, quarterly, ad hoc) which exacerbates the planning issue. They also have completely different communication methods/styles – github, jira, mailing list, private support contracts, etc.
A lot of devices now have an OS, so you could have unexpected issues – light bulbs3, smart TVs, your home routers, printers4.
Addressing this at an organizational level takes some commitment. For instance, having some guidance for security requirements, similar to performance testing. Like performance, applying these requirements does require some judgement.
It’s worth noting that code on stackoverflow, or a forum post that is copied into your application could be used in the same fashion as a third party library, but would be missed by some classes of scanning tools. Similarly, code that is built from source would need extra maintenance (although perhaps issues could be found through the use of static analysis or black box testing). It seems to me that this would make stackoverflow posts / blogs / forums good places for bad actors to own. This is often the case for “free” WordPress themes and plugins.
A list of useful tools was also mentioned:
After seeing this, I think it might be interesting to research additional build tooling for developers, e.g. searching for vulnerable maven dependencies from an SBT / webpack plugin. I also wonder how many browser plugins steal tokens or inject code (how would you even tell?), so clearly there is a lot of interesting happening.