Taking into account the increasing need for online data protection and the availability of numerous Privacy Enhancing Technologies (PETs) for internet and mobile users, ENISA conducted, under its 2015 work programme, a study on online privacy tools, aiming at enhancing trust and assurance in their use by the general public. In particular, this study includes three parts: a) a review of existing web portals promoting the use of online privacy tools for the general public, b) a proposal for a methodology for evaluating the quality and functionality of PETs, and c) a pilot application of the proposed methodology in the area of anti-tracking browser extensions. The review analysed several web portals that are listing and/or recommending the use of specific online privacy tools (e.g. for secure messaging, anti-tracking, encryption, etc.). The main focus was on the criteria applied for the selection (and further assessment) of these tools. The analysis showed that there are no commonly accepted methodologies and in most cases there is not even a description of the rationale used for selecting and recommending certain tools. To this end, based on the results of the privacy portals' review, a number of criteria was defined relating to the reliability and usability of online privacy tools, especially when targeting the general public. These criteria set the basis for ENISA's proposal for a generic PETs evaluation methodology and are divided in three categories: basic, quality and functionality. The basic criteria are a preliminary set of fundamental characteristics that the tool should have (a threshold for proceeding with further analysis). They are related to the maturity and stability of a PET, overall maintenance (last update), reactivity to known vulnerabilities, ease of access and installation, availability of documentation. The quality criteria aim to assess generic quality features related to the reliability and usability of the tool. In that respect they include: background information for the tool, version history, transparency of installation and use, available public reviews, privacy by design and by default, ease of use, user interaction and side effects. Last, the functionality criteria assess whether the tool offers the promised functionality and features (in other words what the tool does and what it does not) and they differ for each area of privacy tools. Following the proposed methodology, as part of the study, a pilot analysis of six popular anti-tracking browser extensions was also conducted. The results of the analysis is a comparative presentation of the tools, aiming at highlighting their differences and similarities and drawing relevant conclusions.
