The Best Open Source Intelligence (OSINT) Tools and Techniques

Intelligence Fusion
10 min readDec 20, 2018

--

This article breaks down the best OSINT tools, techniques, resources and websites available online for every stage of intelligence gathering process. From background reading, to organising your research and getting the best out of search engines, Intelligence Fusion has created the ultimate list of open source intelligence tools.

We’ll cover the following;

  1. What is OSINT?
  2. The Best OSINT Websites for Background Reading
  3. OSINT Techniques to Organise your Thoughts
  4. Setting up Automated Alerts using OSINT Tools
  5. OSINT Tools for Social Media
  6. OSINT Techniques: Getting the Best out of Search Engines
  7. Platforms and Dashboards for Open Source Intelligence
  8. OSINT Websites for Online News Media
  9. Other Useful OSINT Tools

What is OSINT?

Open source intelligence, or OSINT, is the collection and analysis of information that is gathered from public, or open, sources.

The Best OSINT Websites for Background Reading

1. academia.edu

This is a platform used to share papers, monitor their impact, and follow the research in a particular field. Academia.edu currently has over 21 million papers.

2. CORE (COnnecting REpositories)

CORE promotes free and unrestricted access to research outputs from repositories and journals worldwide. The public can search through a collection of over 125 millions of harvested research outputs which can be downloaded free of charge.

3. Library of Congress

The Library of Congress is the largest library in the world, with millions of books, recordings, photographs, newspapers, maps and manuscripts in its collections.

4. Open Knowledge Maps

Open Knowledge Maps presents you with a topical overview of your search term. It’s based on the 100 most relevant documents for your search term, which allows you to easily identify useful, pertinent information.

5. WolframAlpha

WolframAlpha is an online service that answers factual queries directly by computing the answer from externally sourced “curated data”, unlike a search engine that provides a list of documents or web pages that might contain the answer the searcher is looking for.

6. BASE

BASE is one of the world’s most voluminous search engines especially for academic web resources. BASE provides more than 120 million documents from more than 6,000 sources. You can access the full texts of about 60% of the indexed documents for free.

7. Google Scholar

Google Scholar is a freely accessible web search engine that indexes the full text or metadata of scholarly literature across an array of publishing formats and disciplines.

8. Disputed Territories

This map identifies the locations and territories that are claimed by more than one country or occupying force.

OSINT Techniques to Organise your Thoughts

1. Microsoft OneNote

Microsoft OneNote is a free-form information gathering and multi-user collaboration. It gathers users’ notes, drawings, screen clippings and audio commentaries and allows your OSINT resources to be easily shared amongst other users online.

Setting up Automated Alerts using OSINT Tools

1. Google Alerts

Google Alerts is a content change detection and notification service that can be used as an open source intelligence tool. It will send emails to you when it finds new results matching your specified search term.

2. Talkwalker Alerts

Talkwalker is an alternative free alert system to Google Alerts that offers a higher level of customisation on alerts and notifications. Talkwalker can also monitor social media mentions of your specified search term.

OSINT Tools for Social Media

1. Seek A Tweet

Seekatweet is designed to help you find relevant tweets in a specific location. You can use their tailored filtering as an OSINT tool to find Tweets near you or anywhere else in the world.

2. One Million Tweet Map

The one million tweet map is a social media tool that you can use to visualise tweets and aggregated Twitter data in a world map. It can be useful when conduction real-time OSINT investigations.

3. Tweeplers

Tweepler is a real time map of tweets coming from various locations. The tweets shown on the map are only the fresh tweets. They are displayed as soon as received. No tweet location is older than a fraction of a minute.

4. Hashatit.com

Hashatit.com is a social search engine for hashtags. You can search tags across all social media platforms, bringing everything relevant to your OSINT research to one place in real time.

5. Mulpix

An advanced Instagram search engine, Mulpix makes searching Instagram as an OSINT technique even easier through the leverage of searching via multiple hashtags.

6. Picodash

Picodash is another Instagram search engine OSINT tool to help you to search and analyse Instagram content by location and hashtags.

7. Echosec

Echosec is a location-based social media geofencing platform that analyses data in real-time to support retail, journalists, marketers and security teams as an open source intelligence tool.

8. TweetDeck

TweetDeck is a social media dashboard application for management of Twitter accounts. OSINT analysts can create live feeds of tweets to monitor specified hashtags, handles or mentions.

OSINT Techniques: Getting the Best out of Search Engines

1. Carrot²

Carrot² is an open source search results clustering engine. It can automatically cluster small collections of documents, for example search results or document abstracts, into thematic categories

2. Newslookup.com

Newslookup.com is a news search engine, news headline, news feed and news services provider that crawls several thousand news media sites providing time based live run down of headlines by region, topic or person.

3. Cluuz

Cluuz is a search engine that can be used as a really useful OSINT tool. Cluuz shows not only links to related pages, but also entities and images that are extracted from within the search results. In addition to the results, Cluuz displays a tag cloud of the most relevant entities extracted from returned results.

Platforms and Dashboards for Open Source Intelligence

1. HealthMap

HealthMap brings together disparate data sources to achieve a unified and comprehensive view of the current global state of infectious diseases.

2. Investigative Database

The Investigative Database allows open source analysts to browse their global index of public registries for companies, land registries and courts as well as search millions of documents and datasets, from public sources, leaks and investigations.

3. IBM Watson News Explorer

The IBM Watson News Explorer automatically constructs a news information network and presents large volumes of news results in an understandable fashion.

4. Human Trafficking Search

HTS produces a weekly blog, publishes research, and hosts a global resource database on human trafficking and modern-day slavery.

5. Marine Traffic

MarineTraffic is a provider of ship tracking and maritime intelligence. Monitoring vessel movements to build a base of data gathered from our network of coastal AIS-receiving stations, supplemented by satellite receivers.

6. Iris

Iris is an application developed by the World Customs Organisation that monitors open source information and presents this information in a graphic-style world map in real-time.

7. ICC Live Piracy Report

This live map from ICC shows all of the piracy and armed robbery incidents reported to IMB Piracy Reporting Centre.

8. ACLED

ACLED is a realtime data and analysis source on political violence and protest. As an OSINT tool, analysts and researched can use ACLED for the latest reliable information on current conflict and disorder patterns.

OSINT Websites for Online News Media

1. NewsNow

NewsNow aims to be the world’s most accurate and comprehensive world news aggregator, bringing you the latest global current affairs headlines

2. Allyoucanread.com

AllYouCanRead.com is the largest database of magazines and newspapers on the Internet, with listings for about 25,000 magazines, newspapers and top news sites from all over the world.

3. Newspaper Map

Newspaper map is a free webapp that helps you find over 10,000 of the world’s online newspapers, and in many cases get to their Google translations in one click.

4. Distill Web Monitor

Monitor webpage or feed for changes asn an open source intelligence tool. Distill runs in your browser to check monitored pages for changes and can send you instant alerts via SMS or email as soon as a change is detected.

5. Feedly

Feedly is a news aggregator application that can be used to compiles news feeds from a variety of OSINT websites, online sources and allows the you to customise and share with others users online.

Other Useful OSINT Tools

Additional tools that can be used as part of an OSINT framework can be found below.

As part of our open source intelligence course, we provide extensive training on the most effective way to use these OSINT tools as well as an impressive and diverse range of additional modules, all built using real-life experiences from our ex-military and intelligence expert team.

1. IntelTechniques and Image Search

2. Tineye

3. Invid

4. Maltego

5. Way Back Machine

6. Transparency International

7. Globalsecurity.org

8. Naval Open Source Intelligence

9. TerraServer

10. Relief Web

11. Tripadvisor

12. Police Sites

13. CIA Factbook

14. FBI Vault

15. OSAC Report

16. Global Terrorism Database

17. Bellingcat

18. Reddit

Intelligence Fusion uses these sources, teamed with military-grade processes and structure to enable a new way of thinking, encouraging analysts to adapt alternative mindsets to better discover, evaluate and understand threats of every kind.

Our OSINT training course can be adapted to clients’ requirements and tailored to meet bespoke requests. We have the capacity to conduct face to face training, across the globe, or alternatively, we can also deliver the open source ntelligence course remotely.

If you’d like further insight into what our training programme consists of, you can download a selection of slides from our standard training modules by clicking here.

Originally published at www.intelligencefusion.co.uk.

--

--

Intelligence Fusion

A threat intelligence data and software provider, helping security professionals across the globe better protest their people, assets and operations.