What DATA is Open Source DATA?
As stated in the previous article “open source” refers to data that is readily available for public consumption. Rather than coming from a single location, OSD can be taken from a range of sources for use in OSINT investigations. It is not enough, however, to simply think of using OSINT. It must be done with intelligence and care. The internet is a highly valuable data source, but reliance on manual investigation methods risks overwhelming analysts and investigators with irrelevant or low-quality materials, leading to wasted time, inefficiency and poor outcomes. Most people or organisations will simply think of using search engines like Google and Bing. However, reliance on these platforms has its downsides: search engines are designed for consumer use, not to answer investigative questions. Using simple search engines for OSINT investigations can limit and skew your findings, as well as presenting a security risk.
Some of the places OSD data can emerge from include:
News media content
Content produced, published or broadcast — including online — for general public consumption in multiple media formats such as journals, newspapers, radio and television. This also includes media platforms that do not necessarily publish original content.
This refers to materials and information from non-media organisations and institutions
• Academic institutions, think tanks and research institutions — for example, academic papers.
• Government agencies — includes information that can be accessed on request, such as
• Businesses and corporations — this would include annual reports and company filings.
• Intergovernmental organisations — reports from organisations like the United Nations and World Health Organization (WHO).
• Charities and Non-governmental organisations (NGOs).
As ever more people increasingly choose to share their lives through social media, social media intelligence (SOCMINT) is particularly important. SOCMINT essentially involves intelligence across two data categories -
1. Original content: such as Facebook updates/uploaded images/video uploads.
2. Metadata associated with original content: Multimedia/geo-location/date/time.
When integrated with other types of publicly available information, or OSD, the insights derived from these information sources enable organisations to resolve complex problems, ranging from financial crime, to due diligence, to fraud and more. The use of external intelligence such as SOCMINT can allow organisations to identify networks of people wanting to harm, steal or defraud the organisation.
Where publicly available, this includes information in both long-form, e.g., blogs and sources such as Reddit, and short-form, e.g., posts on Facebook, Twitter, and LinkedIn.
Multimedia instant messaging apps enable people to share pictures and videos with their social networks around the globe. Some multimedia instant messaging apps incorporate a feature that enables users to share content that self-destructs after a short period of time, making it inaccessible to recipients after this period.
Self-destructing messages permit criminals an opportunity to advertise illicit goods for sale, like drugs and weapons, in a way that limits detection by law enforcement agencies. Billions of multimedia messages are sent via these services each day, with many available for public consumption.
Digital Footprinting enables visualisation and analysis of a subject's online relationships and behaviour in a clear and easy-to-understand manner.
With the growth of social media users, the spread of the Internet, and the proliferation of mobile devices, digital footprints represent increasingly accurate pictures of individuals. While digital footprints usually encompass a variety of online activity such as posts on social networks, likes, comments, stories, geolocation points, etc., even an inactive social media presence, such as the mere existence of a profile, can prove connections between people and becomes reliable evidence in the fight against fraud.
The dark web is a treasure trove of data often linked to criminal activity. It can contain data such as usernames, email addresses and phone numbers of individuals connected to crimes. Extracting and using data from all the corners of the deep and dark web can enrich investigations and enable more effective recommendations to clients.
SL ISE and Facial Recognition
Involves using Social Links' Identity Search Engine to discover less mainstream data sources and take advantage of an advanced AI-driven image recognition tool.
is the analysis of location imagery and information to assess and analyse physical features and geographical activities on the Earth. Geospatial OSINT is the collection and analysis of publicly available geospatial data.
Geospatial OSINT techniques are conducted by government agencies, law enforcement agencies, and private companies, with research conducted by investigators, analysts, journalists, and other researchers. These techniques can be applied during investigations to assist in the location of subjects and missing persons and to research global events like pandemics and natural disasters.
is information created inside a business. External data can come from numerous other sources, including news media, search engines, social media and corporate records data. To ensure that all available data is being utilised (and that no potential intelligence is being missed), investigations tools that integrate external and internal databases make it possible to recognise hidden connections and highlight suspicious behaviours across disparate data sets.
More sources and more possibilities
In the past, open-source material was mostly limited to printed media, such as books, articles and public records, that could only be viewed at specific places and times. Online data provides on-demand access to published material, as well as self-published blogs
and social media posts. There is also now a whole range of visual and auditory media that did not exist prior to the development of smartphones and mobile technology.
While providing professional researchers and investigators with larger volumes of potentially
useful information, the rapidly expanding nature of OSD also threatens to overwhelm. Therefore, applying rigour to the way in which OSD is collated, analysed and used is now more important than ever.
In many ways, the Information Age is also proving to be the Age of OSINT.