2

I'm attempting to improve my understanding of the DNS system and am confused as to how domaintools.com manages to resolve all DNs associate with an IP addr.

After playing around with dig, i'm stuck.

Example on domain dressupgames.com

using dig::

$ dig +short dressupgames.com                                                                                      
173.231.156.28

$ dig +short -x 173.231.156.28                                                                                     
28.24/29.156.231.173.in-addr.arpa.

$ dig +short 28.24/29.156.231.173.in-addr.arpa
67.215.65.132

$ dig +short -x 67.215.65.132                                                                                      
hit-nxdomain.opendns.com.

$ dig +short hit-nxdomain.opendns.com
67.215.65.132

Presumably 173.231.156.28 is an alias for 67.215.65.132 (?). Now using DomainTools web interface http://www.domaintools.com/research/reverse-ip/ i get the promise of 129 results.

How can i find these results using commandline tools?

1
  • 7
    You're using some feature of OpenDNS which redirects nonexistent DNS lookups to their own servers rather than returning NXDOMAIN. You can turn that off. Oct 30, 2012 at 16:03

2 Answers 2

6

DNS doesn't contain an automatic IP to hostname lookup mechanism. It does have PTR records, but these are configured manually and don't correspond with the A records in most cases. Sites like domaintools.com simply have a big database of all the forward mappings they've discovered and they query this when an IP is looked up. The results are not guaranteed to be complete or up to date.

1

I'm the creator of host.io, which also shows you a list of all of the domains hosted on the same IP address (along with a list of domains that link to the domain, and more). For example, here's a list of domains hosted on the same IP as stackoverflow.com: https://host.io/stackoverflow.com

As you've discovered, getting the IP address for a single domain is only a very small part of the solution. There is no single command or script you can write to do this - you need to build out your own database of domains to IP address, which is exactly what DomainTools, us, and others like us do.

First need to get (or create) a list of all available domain names. There are roughly 250 million currently. The next step is to resolve all of those domains to an IP address. You then need to store all of those domain to IP pairs in a database, and then you can query to get a list of all domains on the same IP. And then you need to do that at a regular frequency to make sure it stays up to date.

To give a full example, let's create a file with 4 domains and resolve them to IP addresses:

$ cat domains.txt
facebook.com
fb.com
stackoverflow.com
stackexchange.com

# Let's resolve the domains to IPs with dig - could use nslookup or similar
$ cat domains.txt | xargs -I% bash -c "dig +short % | tail -n1" > ips.txt
31.13.76.68
31.13.76.68
151.101.129.69
151.101.193.69

# Let's combine the domains and IPs using paste
$ paste domains.txt ips.txt > combined.tsv
$ cat combined.tsv
facebook.com    31.13.76.68
fb.com  31.13.76.68
stackoverflow.com   151.101.129.69
stackexchange.com   151.101.129.69

# Let's create a DB table and import the data, and write a query 
# to find any domains in our dataset that are hosted on the same 
# domain as stackoverflow.com

$ psql $DB_URL

=> create table details (domain text, ip text);
=> \copy details from ~/combined.tsv;

=> select domain from details where ip = (select ip from details where domain = 'stackoverflow.com');
      domain
-------------------
 stackoverflow.com
 stackexchange.com
(2 rows)
4
  • 1
    I'd note that an additional challenge is that many domains have different IPs depending on where the requesting client is located geographically. If I dig amazon.com from New York City it's going to be different than if I do it from Australia.
    – ceejayoz
    Nov 7, 2018 at 19:43
  • There isn't any way to acquire a complete list of existing domain names. Even specifying how many there are isn't feasible as it isn't well defined what it means for a domain name to exist.
    – kasperd
    Nov 7, 2018 at 22:59
  • Oh really? domainlists.io Nov 8, 2018 at 2:06
  • @BenDowling Not all TLDs provide data feeds of registrations. That one does appear to be pretty comprehensive, but it still won't be a complete list. Good enough for many purposes, sure.
    – ceejayoz
    Nov 8, 2018 at 13:50

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .