Click here to Skip to main content
65,938 articles
CodeProject is changing. Read more.
Articles / Languages / Python

Digital Footprint Mapper - DFMAP

4.90/5 (3 votes)
19 Aug 2014CPOL4 min read 13.5K   205  
Know you digital presence / cyber presence / internet presence using DFMAP

Downloads

Overview

Digital footprint mapper (DFMAP) is a fully automated tool which helps in identifying the footprints of the organization on the internet. Digital foorprints have critical piece of information which can be leveraged by cyber criminals, competetors and by the organization itself for tracking and being well-informed about their digital presence. Hence it becomes significantly important for the organization to know what infomation are they sharing / leaking on the internet. DFMAP carries out passive and semi passive scans to identify internet presence / digital presence of the organization.

Digiatal presence is also very critical for the organization due to the below mentioned points:

  • As persistent attacks on internet are growing exponentially, an organization should be well-aware of the type of information being disclosed on the internet. A sensitive information which is leaked over the internet may be used by the anonymous users to compromise the assets of the organization, cause reputational damage to the organization etc.

  • It would be helpful in performing the domain management, inventory management and also to manage you presence over the internet in a better way.

  • It would give the sentiments of the people and what is being talked about the organization over the internet.

  • No prior approvals are required to carry out the internet presence as it performs a passive and semi passive reconnaissance, it can be used to get the insights of the competitors by performing their digital presence / internet presence.

Functionalities

Below mentioned are the supported functionalities in DFMAP :
  1. Search engine scrapping (google, yahoo, bing, ask, lycos)
  2. Robtex search (All identified domains from searching engine scrapping undergoes robtex search.)
  3. Domain to host/IP mapping
  4. Whois on domain
  5. Whois on IP
  6. Metadata extraction of the dowloaded files (pdf, xls, doc etc) (Publicly available files would be downloaded upon which exiftool would be used to extract meta data from the documents )
  7. Fingerprinting of domain (to identify the remote OS in a passive way, p0f tool is being used. If you choose to execute this module then please make sure you have p0f-3.06b installed on your machine. This module will require you to pass the complete path of the file which stores the output of p0f. For example under dir/home/john/tools/p0f-3.06b you run the tool p0f by typing "./p0f -i eth0 -o sample-capture.txt" then the input for the module would be the complete path of the sample-capture.txt file which in this case is /home/john/tools/p0f-3.06b/sample-capture.txt)
  8. Google dorking (Choose your query wisely under conf/dorkqueries.txt as it takes quite a while to execute all the queries. By default the number of queries would be 2500+)
  9. Shodan (Apart from the enteries in the database, detailed .txt file can be found under output/shodan-results.txt)
  10. Unstructured data analysis using nltk package
  11. SSL certificate validation
  12. Technology statistics 

Dependencies

Following are the dependencies to execute the digital foorprinting program.

  1. Python 2.7.5+
  2. MySQL database
  3. Python MySQLdb package
  4. webarticle2text
  5. nltk
  6. nltk.download()
  7. BeautifulSoup

Screenshots

1. Following screenshot shows the extracted meta data like author name, from the publicly available documnets like pdf files etc.

Image 1

2. Following screenshot shows that enteries captured by extracting meta data from the publicly available documents.
Image 2
3. Following screenshot shows domain to IP mapping, of the unique domains identified from search engine scrapping.
Image 3
4. Following screenshot shows the list of unique url's identified on various domains.
Image 4

5. Following screenshot shows that the enteries of whois information for the identified unique domain.

Image 5

6. Following screenshot shows that the enteries of whois information for the identified unique domain.

Image 6

7. Following screenshot shows that fingerorinting information of the domain names like web server in use, operation system in use etc.

Image 7

8. Following screenshot shows that the enteries of whois information for the identified ip address (which host the domains).

Image 8

9. Following screenshot shows that the enteries of whois information for the identified ip address (which host the domains).

Image 9

10. Following screenshot shows the enteries of the ssl certificate parameters like validay of the certificate, issuer, subject etc. identified on various domain.

Image 10

11. Following screenshot shows the enteries of the ssl certificate parameters like validay of the certificate, issuer, subject etc. identified on various domain.

Image 11

12. Following screenshot shows technologies in use on various domain.

Image 12

Using DFMAP

Reference:

A detailed video is available here.

Github souce is available here.

//
// Code can be used by typing the following command
#  python profiler.py

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)