This repository contains the known hosts blocklists that are made available to the users of AdGuard products (AdGuard DNS, AdGuard Home, etc).
Some of these blocklists are automatically converted to the rules format that AdGuard product understand better.
filters.json contains all the blocklists added to the repo. downloadURL is the location of the re-hosted blocklist.services.json is the meta-data of “Web Services”. This is a part of the parental control functionality of AdGuard Home and AdGuard DNS./etc/hosts can also be added if there is no alternative.The blocklist should have a clear purpose.
Examples:
To add a new filter, you need to:
filter_(id)_(filter name).configuration.json and metadata.json. The contents of these files are described below.locales/en/filters.json add an object with the name and description of the filter according to the example added above.The files /assets/filters.json and /assets/filters-dev.json must not be edited manually.
metadata.json
Filter metadata. Includes name, description, etc.
filterKey — unique human-readable filter identifier (string)filterId - unique internal filter identifier (integer)name — filter name; can be localizeddescription — filter descriptiontimeAdded — time when this filter was added to the registry; milliseconds since January 1, 1970; you can exec new Date().getTime() in the browser console to get the current timehomepage — filter website/homepagedeprecated — optional, boolean, filter is deprecated and should not be used by the products,
but filter is still available, i.e. not removedexpires — filter’s default expiration perioddisplayNumber — this number is used when AdGuard sorts available filters (GUI)environment - either dev or prod. Only prod lists are available in AdGuard DNS.disabled - if set to true, the blocklist won’t be updated.tags — a list of tagstrusted - a flag that allows using $dnsrewrite rules for this filter. If the filter is not trusted, $dnsrewrite rules will be removed from the compiled filter.
{
"filterKey": "adguard_dns_filter",
"filterId": 1,
"name": "AdGuard DNS filter",
"description": "Filter composed of several other filters (AdGuard Base filter, Social Media filter, Tracking Protection filter, Mobile Ads filter, EasyList and EasyPrivacy) and simplified specifically to be better compatible with DNS-level ad blocking.",
"timeAdded": 1404115015843,
"homepage": "https://kb.adguard.com/general/adguard-ad-filters",
"expires": "4 days",
"displayNumber": 3,
"environment": "prod",
"tags": [],
"trusted": true
}
</details>
revision.json
Filter version metadata, automatically filled and overwritten on each build.
filter.txt
Resulting compiled filter.
configuration.json
Configuration defines your filter list sources, and the transformations that are applied to the sources. See Hostlist compiler configuration for details
Every filter can be marked by a number of tags. Every tag metadata listed in /tags/metadata.json.
Possible tags:
lang:* — for language-specific filters; one or multiple lang-tags can be used. For instance,
AdGuard Russian filter is marked with the lang:ru tag.
purpose:* — determines filters purposes; multiple purpose-tags can be used for one filter list.
recommended — for low-risk filter lists which are recommended to use in their category. The
category is determined by the pair of the lang:* and purpose:* tags.
obsolete — for abandoned filter lists; filter’s metadata with this tag will be excluded
from filters.json and filters_i18n.json.
To add a new blocked service, a new .yml file must be added to the services directory, the contents of the directory and the resulting services.json file are described in the next section.
The services.json file combines information about services received from separate .yml files located in the services directory. The services.json file is generated by a script, so there is no need to modify it manually.
Source .yml files: Each .yml file contains information about a specific service. The file name must match its id.
File saving: .yml files are stored in the services directory.
Updating information: In cases when it is necessary to change the configuration of a service, it is necessary to update the corresponding .yml file in the services directory.
Adding a new service: to add a new service to services.json it is necessary to add .yml file to the services directory, after the script is run the file will be updated.
WARNING!
Deletion of files is strictly forbidden. If a service file is deleted, the script will restore it from a previous version of the services.json file.
It is also forbidden to change service id inside the .yml file.
Service metadata includes id, name, rules, icon_svg:
id — unique human-readable service identifier (string, as snake_case)name — service name (string)rules — list of domain rules in Adblock syntax (list of strings)icon_svg — svg icon (string)SVG icon requirements:
viewBox attribute must be equal.fill="currentColor" attribute. This is important for themes to work properly.width and height attributesengines_safe_search.txt contains a set of rules to enforce Safe Search in the following searching engines:
youtube_safe_search.txt contains a set of rules to enforce Safe Search and hide comments on YouTube.yarn install
Run the following command:
yarn run compose
The build result can be found in the assets directory.
Once a year, we will compress the repository to reduce its size. We will delete all remote branches and overwrite the main branch with a squashed history. The compression script will retain the first N commits in their original order in the history. All other commits (except the first one) will be squashed into a single commit.
yarn install
yarn compress [commits_to_keep]
It will retain the first [commits_to_keep] (default is 10000, which is approximately one year of history) commits, starting from now, in their original order in the history. All other older commits (except the very first one) will be squashed into a single commit.
git push --set-upstream origin --force main
git ls-remote --heads origin
Remove remote branches that are no longer needed locally and push the removal to the remote repository:
git push origin --delete branchName
Replace branchName with the name of the branch you want to delete.
Use git remote prune origin to remove references to remote branches that have been deleted on the remote repository. This keeps your local repository in sync with the remote:
git remote prune origin
Over time, Git can accumulate references in the reflog that are no longer needed. You can clean the reflog using the following command:
git reflog expire --expire=now --all
git gc --aggressive --prune=now
This will remove unnecessary entries from the reflog and perform garbage collection.
After this procedure git repository will reduce it’s size.
Blocklist names, descriptions, and tags’ names are translated via crowdin.
/locales contains translations for filters, groups, and tags.
Base language strings:
locales/en/filters.jsonlocales/en/tags.jsonIn order to prepare these files run yarn run locales:prepare.
This script will scan filters meta and add filters names, descriptions, and tags meta to the base language files.
yarn run locales:uploadyarn run locales:download