7.2. Network Discovery

The Network discovery program (sometimes referred to as the "Network Crawler") needs a host to start from. This host is called the "seed host"; you enter it in the first page of the Druid (Figure 7-10). The crawler implements the following algorithm (this is a somewhat simplified explanation):

First, it runs several SNMP queries against the seed host trying to collect the list of its interfaces and its ARP and routing tables. This host is then added to the table of discovered network objects, together with its interfaces, their addresses and netmasks, and the host's "sysinfo" parameters. Then the crawler analyses the routing table of that host; this allows it to discover the networks and subnets, which in turn are also added to the list of discovered objects. Then it analyses the ARP table, which holds MAC and IP addresses of neighboring hosts. It takes one host at a time from this table and repeats the same algorithm, using the new host as a seed host. When it pulls an ARP table from the next host, it discards entries that describe objects it already knows about. However if it finds new entries, it tries them as well and thus travels further down the network. Eventually it will visit every host on all subnets on the network.

This algorithm relies on hosts answering to the SNMP queries. If the very first host (the "seed" host) does not run SNMP agent, the crawler will stop on the first run of its algorithm and won't find anything. Therefore it is important to use a host which does run SNMP agent as a "seed" host. Even if most of the hosts on the network do not run SNMP agents, but a few do, the crawler will most likely find all of them. This happens because it discovers objects when it reads the ARP tables from the host which answers; so even if discovered hosts do not answer to SNMP queries, the crawler already found them anyway.

One of the ways to limit the scope of the network that the crawler will visit is to use the "Confine scan to the network" parameter. You need to enter both a network address and a netmask; the crawler will then check if the hosts it discovers belong to this network and if they do not, discard them.

There are a few settings that affect the crawler's algorithm (see Figure 7-11 and Figure 7-12). Here is the list:

• Run network scan recursively

  As was described above, the crawler starts with the "seed" host and then repeats its algorithm using every discovered host as a new "seed". If this option is turned OFF, then the crawler runs its algorithm only once and stops.

• Follow point-to-point links

  If a firewall or router has a point-to-point interface (for example, PPP interface), then the crawler can automatically calculate the IP address of the other side of this interface. It then continues the discovery process by querying a router on the other side. Very often, the point-to-point link connects the organization's network to an ISP and you are not really interested in collecting data about your ISP network. By default crawler won't cross point-to-point links, but this option, if activated, permits it.

• Include virtual addresses

  Sometimes servers or routers have more than one IP address assigned to the same interface. If this option is turned on, the crawler "discovers" these virtual addresses and tries to create objects for them.

• Run reverse name lookup queries to determine host names

  If the host discovered by the crawler answers to SNMP queries, it will report its name, which the crawler will use to create an object in Firewall Builder. However, if the host does not answer the query, the crawler cannot determine its name and only knows its IP address. The crawler can use DNS to back-resolve such addresses and determine host names if this option is turned ON.

• SNMP (and DNS) query parameters

  You must specify the SNMP "read" community string which will be used for SNMP queries. You can also specify the number of retries and a timeout for the query. (The number of retries and timeout parameters also apply to DNS and reverse DNS queries.)

Once all parameters are entered, the crawler actually gets to work, which may take a while. Depending on the size of the network and such parameters as the SNMP timeout value, scanning may take minutes or even hours. The progress of the scanner can be monitored on the page in the Druid (Figure 7-13) and (Figure 7-14). You can always stop the crawler using the "Stop network scan" button. Data does not get lost if you do this as the Druid will use whatever objects the crawler discovered before you stopped it.

The "Save scan log to file" button saves the content of the progress window to a text file and is mostly used for troubleshooting and bug reports related to the crawler.

If the crawler succeeded and was able to collect information it needed to create objects, you can switch to the next page where you choose and create objects. That page is the same as the one shown in Figure 7-2.