Decoding The Deep Web

What is the Deep Web?
If you’re reading this article you’re clearly familiar with the unencrypted, accessible portion of the World Wide Web that is readily available through search engine results. However, while most people think of this as the internet, there is a whole world of additional content hidden away which is less easily found. The public internet is like the icing on a cake, with plenty of hidden layers awaiting discovery.
When it comes to this lesser-known portion of the internet, different people use the same terminology to mean different things. Hidden parts of the World Wide Web are variously known as the Dark Web, the Deep Web, the Invisible Web or the Hidden Web. The most commonly-used collective term is the Deep Web, encompassing pages that don’t show up in conventional search engine results but can still be found with the right software and knowledge.
Many parts of the Deep Web are hidden from public view to avoid confusion. From company intranets and maintenance versions of mainstream websites through to file directories and archived pages, there are often good reasons for content to be cloaked. Secure platforms like online banking sites are a good example of Deep Web content, as are those annoying CAPTCHA screens that require validation when purchasing goods or setting up a new account.
However, the Deep Web also contains innumerable sites that are less benign in their intentions, deliberately buried beyond the reach of Google and Bing. Some morally-outraged people argue that this is a haven for the worst criminal activities imaginable, though in reality it’s not that easy to crowdfund a hitman or buy a kilogram of illegal narcotics. Pornographic content is readily available, but it has to be deliberately sought out, although the public internet is hardly a porn-free zone, either.
Because Deep Web addresses don’t appear on the search engines, finding these sites can be tricky. There are specialist search engines that attempt to illuminate dark corners of the internet, but attempts to create a Deep Web crawler have so far proved largely unsuccessful and wiki sites tend to be populated with dead links. Addresses usually comprise long streams of alphanumeric characters rather than words, and the content itself is rarely hyperlinked from elsewhere – the main indexing method used by conventional web crawlers.
A favoured entry point for Deep Web exploration is the Tor browser, which is based on Firefox and works on Windows, Linux and Mac OS X-powered devices. An abbreviation of The Onion Router, this anonymous connection software bounces traffic through a complex network of relays to make it extremely difficult for an individual’s activity to be traced. Perhaps surprisingly, Tor is funded mainly by the US Government, with the remainder of its budget supplied by other Governments, NGOs and loyal users around the world.
Tor’s multi-layered encryption of data (including IP addresses) at randomised points of each routing process makes it almost impervious to interception or cracking. It would therefore be easy to assume that anything sent or received through Tor must inherently be illegal or immoral. In fact, as a tool for achieving freedom of expression and avoiding covert surveillance, Tor is unparalleled. It’s won awards for allowing people in politically unstable countries to anonymously expose corruption and campaign for democracy, and it is also a haven for whistleblowers like Edward Snowden.
In essence, the Deep Web is only as good or bad as the people who use it. In that respect, it’s rather like the internet we all know and use on a daily basis – but multiplied by a factor of ten.