Imagine you are administering a small office or home network. Perhaps you want to know what hosts in this subnet are currently online, or which service that Internet of Things (IoT) device keeps open to the world. Network scanners are tools built to do just that.

Even if a host is properly secured and has unused ports closed, a network scanner may tell quite a lot about it. There are slight discrepancies in how popular operating systems (OSs) implement network protocols such as TCP. A tool that knows these nuances can make an educated guess about which OS the host runs. This is known as OS fingerprinting, and many network scanners implement it as well. Sometimes, it can even give you an uptime estimate!

As you guessed already, this Core Tech is about network scanning. Before we dive in, a usual word of warning: As with many technologies, network scanning can be used for good and for evil. Many network attacks begin with it, so it is deemed illegal in some provider and corporate networks. Never scan a network you don't really own unless you have permission to do so. When in doubt, a purpose-built scan target, scanme.nmap.org, is a good choice.

Host Discovery

As usual, Linux doesn't come up short of network scanners, and many are free as in speech. Of these, Nmap [1] is perhaps the most ubiquitous. Nmap stands for Network Mapper, and it is (naturally) a command-line tool (actually, a set of tools). For those of us not looking for hacker brownie points, a GUI called Zenmap (Figure 1) is also available.

Figure 1: Zenmap wraps Nmap goodies in a simple yet feature-rich Gtk+ user interface.

Nmap should be already in your package manager, so you don't need to compile it from the sources. Many of the operations it performs require raw sockets or are otherwise privileged, so you typically run nmap via sudo.

Imagine you connect to some IP network and want to know which hosts are online. What should you do? What springs to mind first is to ping each host in the subnet in turn and look for replies. This won't work well for large /8 networks (16M hosts each), but for a typical /24 (256 hosts) or smaller, it's a matter of minutes or seconds. This can be done faster if you ping multiple hosts in parallel. This technique is known as a ping scan, and of course Nmap implements it for you:

sudo nmap -sn 192.168.0.0/24

Figure 2 shows the result. Here, we scan a complete subnet, but Nmap understands many target specifications. It could also be a single host for which you provide either an IP address or DNS name. Although not very useful for host discovery, this target is very common in port scanning, which we will cover next. You can also use IP address ranges: 192.168.0.35-40 or 192.168.0.1,2. Moreover, it is possible to exclude certain targets with --exclude:

sudo nmap -sn 192.168.0.0/24 --exclude 192.168.0.1-10

scans everything in the subnet except the first 10 hosts.

Figure 2: Ping scan results. This kind of output isn't particularly eye-catching, yet it is featured in quite a few movies, including The Matrix: Reloaded.

So far, so good. What's wrong with ping scan? Nothing, actually, except some administrators may block ICMP on their hosts. Blocking is a bad idea as ICMP has more applications than mere pings (aka ICMP Echo), yet it's rather widespread. If you can't know reliably if the host is off or just blocking pings, your next best guess is to use another technique and combine the results.

If the hosts you are interested in are in the local Ethernet segment (i.e., they see the broadcast traffic you send), ARP ping is a good choice. Instead of pinging each of the hosts, you ask them to resolve the corresponding IP into a MAC address. Nobody suppresses ARP in a sane state of the mind, as this renders the host pretty useless on the IP network. This makes an ARP scan quite effective. It's not 100 percent accurate as well (nothing is), but again, combining the results of two scans reduces the error.

$ sudo nmap -PR 192.168.0.0/24

ICMP and ARP pings are not the only host discovery options. Other techniques exploit the fact that transport-level protocols, such as TCP or UDP, define a specific feedback if a remote party tries to access an open or closed port. Say, accessing a closed UDP port results in an ICMP Port Unreachable message sent to the originator. As with ARP, these messages are essential for the normal operation and unlikely to be blocked or filtered, which makes them promising candidates for host discovery.

Nevertheless, network providers can monitor for and block suspicious activity, such as a large number of connection attempts made from a single source IP address within a short timespan. Nmap mitigates this with the -T switch, which can make scan operation less aggressive, thus less suspicious and bandwidth-consuming. Of course, this also means they would run for much longer. -T accepts a single argument, which is either a number in the range of zero to five or a self-explaining keyword: paranoid, sneaky, polite, normal, aggressive, and insane. normal is the default. You can find all the details regarding TCP and UDP pings, as well as the -T option, at [3].

nping: Not Your Grandpa's ping

Everyone knows ping. This ubiquitous tool typically sends ICMP Echo messages and is perhaps the number one way to check if the given host is online. But, as we've learned today, there are many other options.

nping, which comes with Nmap, incorporates these options in one tool. It can send TCP and UDP probes, ICMP and ARP pings, and can also reveal intermediate hops working as a traceroute substitute. Moreover, it provides many options to tweak just about every bit in a protocol header, from Ethernet to TCP. This makes nping not merely a diagnostic tool, but a powerful packet generator you can use for fuzzing, stress testing, and other purposes.

This is how you do a TCP probe for port 80 (HTTP):

sudo nping -c 1 --tcp -p 80 scanme.nmap.org

-c tells nping to send one probe only. The tool reports various packet details, such as TCP flags and sequence numbers.

Note that sending TCP probes requires root privileges.

Port Scanning

Now that you know which hosts are online, what should you do next? Perhaps you want to know which services these hosts run – or at least expose to the rest of the world. And this is not a mere curiosity: If you run a service you intend to be internal (such as a database), you must be sure it's not visible from the outside.

You obtain this information with a technique called port scanning. There are a handful of ways to scan ports, but perhaps the simplest one (and the one requiring no root privileges) is to do a connect() to the port in question. This is no different from what an ordinary client application such as a web browser would do. Nmap calls this a TCP connect scan:

sudo nmap -F -sT 192.168.0.1

The result may look similar to Figure 3, where I scan my home network. -sT prescribes a TCP connect scan, and -F makes it "fast" by scanning fewer ports than Nmap would scan by default. Some ports are open, as they run services my home router provides to the LAN, such as web management interface (80/TCP) and DNS (53/UDP). You can also spot a supposed Windows machine.

Figure 3: Port scan results. This could be lengthy for a busy network with many hosts running services.

Heuristics other than -F are available as well; for example, you can scan the top N most popular ports, according to Nmap's database, with --top-ports. For example, the top five ports in my Nmap installation include 21-23 (FTP), 80 (HTTP), and 443 (HTTPS). This works well for common services and quick checks, but for a deeper understanding, you'd want more control over port ranges. Nmap provides it with -p: This switch accepts individual ports (22) as well as port ranges (22,222, 6881-6889). You can prefix the numbers with U: to denote UDP ports or with T: for TCP ones.

TCP connect scan is easy yet not particularly fast. To establish a TCP connection, the parties must exchange three messages (SYN, SYN/ACK, ACK) commonly referred as a three-way handshake. This is not required for a port scan, as the very first reply from the remote party indicates whether the port is open or closed. A TCP SYN scan is the faster alternative, which sends only the initial SYN packet. If the remote side responds with SYN/ACK, the port is open. If it sends RST, the port is closed. Anything else, including ICMP error messages, is a clear indication that the port is filtered by a firewall, or the target doesn't run a compliant TCP stack, which is quite rare.

With the following, we can scan the default range of ports:

sudo nmap -sS 192.168.0.1

So far, we have seen Nmap output on screenshots in a human-readable format. While this format is most common, it's not the only one available.

If you intend to parse the results with some code, -oX dumps the data into XML. It may seem very 1990s, compared to JSON, but you can easily reference an XSL style sheet (try --webxml) to make it viewable within any modern browser (Figure 4). -oG produces "grepable" output (Figure 5), which makes it easier to use Nmap in shell scripts; you can think of it as of something akin to ip -o. To pipe Nmap output into grep, you can use - as a filename.

Figure 4: Use XML output for parsing – or producing some good looking reports for a customer demo.

Figure 5: In grepable output mode, each piece of data occupies exactly one (long) line of text.