Nmap Book finally released!

I'm really excited. After years of work Fyodor finally managed to finish The Book "Nmap Network Scanning". It should be a great gift for every security geek. You can preorder it on Amazon for 34$.

Presentation: Nmap and asynchronous programming

In this presentation I would like to talk about my adventure with extending Nmap and what I learned from it regarding asynchronous programming.


Nmap is a port scanner with many other features.

On this slide you can see an example of Nmap output.

It shows open tcp ports on some target machine. It also shows service type (if it is ssh or http or something else) and version of a daemon running behind discovered port.

Nmap can execute user defined scripts for various services. Above we can see an example result of a script that retrieves HTML title from the HTTP server.

Nmap also tries to guess the operating system on the target machine. In the example output Nmap OS detection wasn't very successful, the OS guesses aren't accurate.


Nmap isn't just a basic port scanner, it's rather fully-featured network discovery tool. It sends a lot of data to the target hosts and opens a lot of tcp/ip or udp connections to the target. To do this efficiently Nmap must be written in a specific way - in asynchronous manner, using nonblocking sockets where possible.

Some parts of Nmap, like OS detection engine, are extremely sensitive for timings.

Nmap sends a lot of custom built packets. The system API for sending custom packets is called raw sockets. Unfortunately this sockets can't be used to listen for raw packets on the wire. To read packets Nmap uses libpcap (packet capture) library. Libpcap is a part of tcpdump (graphical interface is known as Wireshark).

When a program is opening pcap descriptor to sniff the network, it passes a BPF filter to receive only the packets that match the filter. The BPF filter is something like regular expression for packets. The filter is compiled and matched against all the packets that come through the wire. This works extremely fast for a single pcap socket, but it's not really created to handle thousands of open pcap descriptors. Every packet that will come to your machine will be matched against every open pcap descriptor, so opening too many pcap sockets may slow down the whole system.

Before going on, please make sure you know what a three-way handshake on tcp/ip connections is.


Every Nmap scan is created from various stages.

First Nmap must know if the target hosts are really online. This is especially useful when you're scanning a wide range of ip addresses. We really don't want to waste time waiting for black-hole ip's. The detection if a host is up can be done using simple icmp ping packet, or using more complicated techniques.

After we know which hosts are up, Nmap does the real scanning. It sends a lot o SYN packets to discover which network ports are open or closed.

After we know which ports are open, we can start service and version detection phase. Nmap opens a lot of tcp/ip connections to detect what services and exactly which daemons are running there.

After this, Nmap can execute operating system detection, and try to guess which OS is running on a target host.

And finally Nmap can execute Nmap Scripting Engine, to grab even more detailed information about running services on open ports. In the first slide we've seen the results of a script that printed a title from html site grabbed from open http server.


Most of people don't understand how Nmap's basic -sS SYN scanning works. This is really a port scanning kindergarden, so if you know how it works, you can skip this slide.

After the host discovery phase we know that a target host is online. Then Nmap tries to determine which tcp ports are open. To do this, Nmap sends a lot of SYN packets to target host via raw socket bypassing Operating System's networking stack.

If the remote port is open, scanning host receives SYN+ACK packet. But the tcp/ip connection wasn't actually established from scanning host OS – Nmap sent the SYN packet, not OS. OS doesn't recognize the SYN+ACK response packet, so it sends RST packet which tells a target to drop a connection.

Why to do scanning this way and not just open a lot of normal operating system tcp/ip connections? Normal tcp/ip connection would need to be closed gently (FIN flag), using even more packets. This SYN scan is faster, with only 3 packets needed to know if a port is opened or closed.


I prefer this diagram over the previous slide. I would like to repeat that nmap injects first SYN packet on the wire. Than nmap sniffs for SYN+ACK packet on the wire.

Of course Nmap can scan many ports at the same time.


We can say that Nmap first sends a lot of SYN packets and then it waits for the SYN+ACK or RST packets. If packet is received from the wire, than Nmap does something with it. For example updates data structures with received information. Then Nmap goes back to the main event loop and waits for yet another packet from the wire.


The point is that every Nmap stage is waiting for something different.

Host discovery phase can wait for icmp ping response packets or many other types.

Port scanning, as I mentioned in previous slides, is waiting for SYN+ACK or RST packets.

Service detection on the other hand is using a lot of normal tcp/ip or udp connections.

OS detection engine is waiting for a lot of magic packets.

Scripting Engine is also waiting for normal tcp/ip or udp connections.

Let's now talk about Scripting Engine in details.


Nmap Scripting Engine (NSE) is a new feature in Nmap. NSE was created because Nmap Service and Version detection wasn't accurate enough. Service detection is really very basic technology. It just grabs a banner from an open port and matches it against thousands of regular expressions.

This is enough for detecting most daemons, but it's not working for more complicated things. For example Skype client opens tcp port. This port returns valid answers for http queries, so it was detected as a http server. But Skype is not an http server. We weren't able to distinguish that before NSE.

The technology behind scripts is not complicated. A script is written sequentially in LUA – convenient simple higher level scripting language.

After normal scanning Nmap discovers which scripts should be executed and runs them against open ports. A script usually is opening a network socket and it writes/receives data from it. If a script is waiting for something on a network socket, it's froze and suspended. When we receive something from a socket, we continue the script until it's suspended again (or exits).

This is called cooperative multitasking and it allows us to write scripts sequentially and execute a lot of them concurrently. Because all the scripts are mostly waiting for some data on a socket.

The scripting engine is based on Nmap asynchronous library NSOCK. This library is capable of waiting for many network descriptors at the same time.


Like every asynchronous library NSock has a main loop in which it waits for descriptors. When something happens on a descriptor Nsock executes a specific callback for the descriptor.


Connecting this with LUA engine is pretty straightforward. When something happens on a descriptor, we just continue the LUA script that is waiting for data on this descriptor.


I thought that NSE is great, but writing scripts that can only wait for normal tcp ip/udp connections is boring. I thought that it would be great to do something on lower level, like injecting custom packets into the tcp/ip connections, or listening to raw packets from the target host.

That's why I created a raw sockets/libpcap support for NSE. It turned out to be rather complicated.


Adding pcap support for Nsock was only part of success. The idea of the changes is rather clear, in one event loop we need to wait for both network descriptors and in the same moment for pcap descriptors. Internally it's rather complicated because of libpcap limitations.

As the result when a packet is received from libpcap descriptor, a callback is called. Just like with normal network sockets.


Connecting this to the Lua engine wasn't easy. The main problem is that, as I mentioned before, pcap descriptors are rather heavy. So one pcap descriptor is opened from a script and shared between all the instances of the same script. This means that when a packet is received from a network we don't actually know which Lua thread should be woken.

I needed to create a custom dispatcher of packets, to know which thread to wake up.


Finally I was able to open pcap descriptors and send raw packets from Lua scripts. I think that the possibilities of such scripts are huge. Here are some examples of my scripts.

The first scripts is able to detect which hosts are using tcpdump in the local network. The script tries to determine if a host has a network card in promiscuous mode. I'd be happy to explain how it works, but that's not in the scope of this presentation. We can say that the script sends some custom arp-request packets and waits for arp-reply answers.

The next script is an implementation of Lcamtuf Otrace tool. It's a traceroute-like tool that can go through firewalls. It injects custom packets into valid tcp/ip connection with small TTL values, so that it can record the ip addresses of the routers on the way.


Next script is an implementation of "ping -R" record route option. The record route option is a flag on ip packet that tells the routers on the way to record their ip addresses. The results are more accurate than normal traceroute but there are only 9 slots for ip addresses on an ip packet.

My favorite script is an implementation of another Lcamtuf tool called p0f. P0f is a passive fingerprinting project. Basically p0f creates an operating system fingerprint from a single SYN packet. The fingerprint can tell which os the target host is running without sending any custom packets. It's stealth and passive.

I wanted to implement this tool, but Nmap is an active scanner. The idea was to open normal tcp/ip connection to the target and create a fingerprint based on SYN+ACK response. The results aren't very granular, but they can show many interesting things. In this slide you can see the results from some host. It seems obvious that port 22 is forwarded to another machine than ports 80 and 443.


My latest work is an implementation of an active OS fingerprinting.

I think that nowadays, in the time of limited ip space, more and more ip addresses use port forwarding. It's rather common to see that one open port is forwarded from one physical machine and other ports are from different host.

I thought that it would be great to do an active OS detection for every port, rather than only for host. Normal Nmap OS detection can be only executed per host.

I implemented a bit limited Nmap OS detection algorithm in LUA, so that you can try to detect operating system for every open tcp/ip port.

The results aren't perfect yet. This is because of a timing issues inside NSE engine. But they are more granular than basic p0f results. On this slide you can see the results from the same host as on previous slide. Now it's obvious that port 22 is from forwarded from Linux box, and port 80 and 443 are from Windows machine.


Let's sum things up.

I created a support for raw sockets/libpcap for NSE. This was intended as an additional feature, but it was possible to implement one of the most complicated parts of Nmap using this feature - an OS detection.

One could ask if it would be possible to implement all Nmap in Lua?

Well. First let's talk about some other issues.

Nmap has different event loop for every scanning stage. This is because of historical reasons - programmers didn't know how to wait for both network sockets and many pcap descriptors in one event loop. Libpcap API is rather complicated, and it's not easy to mix it with normal sockets.


The problem I described can be generalized. Not only Nmap faces this issue.

The point is that asynchronous programming fails if anything else than event loop is blocking. In Nmap the problem is with specific libpcap limitations, but other projects also face some blocking elements.

Web servers have problem with blocking stat(2) syscall. They use this call to check if a file was modified on the disk and needs to be reloaded. For example lighttpd uses a lot of complicated code to address this issue. Basically lighty opens many threads, one for every physical disk, in which it does blocking stat calls.

Web browsers share similar problem. If you open a file from a slow media, like cdrom, the whole browser stops for a while. This is because open(2) syscall blocks until the metadata is read from the media. I don't like when my browser halts because one tab is slower than the others.

There are also a lot of asynchronous programs where dns queries are done using normal operating system api, which is blocking. In the test environment the software runs perfect, because dns queries are fast, but in production the program fails to scale.


In my opinion asynchronous programming should be easy and fun. Async libraries should hide low level problems from the programmer.

There are a lot of async libraries, but in my opinion there's no complete solution yet.

That’s why my dream is to create yet-another-but-better asynchronous library.

Given such, perfect, async library Nmap could be rewritten in one event loop and possibly in higher level language.


Fyodor, the author of Nmap created a book. It's still in beta, but I hope it will be in shops in few months. I highly recommend it for people interested in low level networking and Nmap.


Thanks! I haven't described how promiscuous node detection works and how my libpcap dispatcher for Lua works. I'd love to write about it some day.



Thanks to Andrzej Teleżyński for proof reading this post.

Nmap - os detection for every open tcp port

While ago I created tool, that shows p0f-like signatures for every open port. This signature is based on information taken from single SYN-ACK packet from target port. Sample output:

$ sudo ./nmap -n -sT -PS80 -p21,22,53,80,443 --script=p0f.nse www.cisco.com

Starting Nmap 4.22SOC1 ( http://insecure.org ) at 2007-07-12 00:04 CEST
Interesting ports on 198.133.219.25:
PORT    STATE    SERVICE
21/tcp  open     ftp
|_ p0f signature: UNKNOWN [65500:61:1:64:M1436,N,N,S,N,W0,N,N,T:AT:?:?] (link: IPSec/GRE, up: 1446 hrs, ipid:54702)
22/tcp  filtered ssh
53/tcp  filtered domain
80/tcp  open     http
|_ p0f signature: UNKNOWN [8192:238:0:44:M1460:A:?:?] (link: ethernet/modem, up: disabled, fill:4008, ipid:1)
443/tcp open     https
|_ p0f signature: UNKNOWN [8192:238:0:44:M1460:A:?:?] (link: ethernet/modem, up: disabled, fill:f8ef, ipid:10)

Now I created similar test, but it does full os detection for open port. It uses nmap os detection algorithms to determine what os is on specific port. This is very useful if target uses destination nat and forwards incoming connections to other hosts. Let's look again at the cisco.com:

$ export NMAPDIR=.
$ sudo ./nmap -sS -p25,53,80,113,443,8080 -n www.cisco.com -PS80 --script=os.nse -O

Starting Nmap 4.53 ( http://nmap.org ) at 2008-02-29 12:51 CET
Interesting ports on 198.133.219.25:
PORT     STATE    SERVICE
25/tcp   open     smtp
|_ os: NCR 5676 or 5688 automated teller machine (94%), Rockwell Automation 1761-NET-ENI Ethernet-to-RS-232-C interface module (94%), Billion 7404VGO-M DSL router (93%)
53/tcp   filtered domain
80/tcp   open     http
|_ os: D-Link DWL-624+ or TRENDnet TEW-432BRP wireless broadband router (93%), Linksys BEFSR41 EtherFast broadband router or D-Link DCS-6620G webcam (93%), HP 9100c Digital Sender scanner (93%)
113/tcp  filtered auth
443/tcp  open     https
|_ os: D-Link DWL-624+ or TRENDnet TEW-432BRP wireless broadband router (93%), Linksys BEFSR41 EtherFast broadband router or D-Link DCS-6620G webcam (93%), HP 9100c Digital Sender scanner (93%)
8080/tcp closed   http-proxy
No OS matches for host
Uptime: 0.007 days (since Fri Feb 29 12:42:05 2008)

I don't suggest that cisco uses D-Link DWL-624. But it's obvious that different host answers to the port 25 and different to ports 80 and 443.

My os detection script is much less reliable than standard nmap -O, because it sends packets to single open port. Normal scan uses also icmp packets, udp probes and tcp closed ports tests.

But I hope that my script can give a brief info of forwarded ports.

If you know how to interpret nmap-os signatures, adding -d1 can give you more detailed information:

$ export NMAPDIR=.
$ sudo ./nmap -sS -p21,22,25,53,80,113,443,8080 -n www.cisco.com -PS80 --script=os.nse -O -d1

Interesting ports on 198.133.219.25:
PORT     STATE    SERVICE    REASON
21/tcp   closed   ftp        reset
22/tcp   filtered ssh        no-response
25/tcp   open     smtp       syn-ack
|  os: Rockwell Automation 1761-NET-ENI Ethernet-to-RS-232-C interface module (94%), Billion 7404VGO-M DSL router (93%), Vodavi XTS-IP PBX (93%)
|      seq info: got packets=6 try=1/1 variance=5.431390
|      ECN(R=Y%DF=Y%TG=40%W=8052%O=NW3NNSM5B4%CC=N%Q=)
|      T2(R=N)
|      T3(R=N)
|      T4(R=N)
|      SEQ(SP=100%GCD=1%ISR=10E%TI=RD%TS=14%uptime=9 minutes)
|      OPS(O1=NNT11NW3NNSM5B4%O2=NNT11NW3NNSM5B4%O3=NNT11NW3M5B4%O4=NNT11NW3NNSM5B4%O5=NNT11NW3NNSM5B4%O6=NNT11NNSM5B4)
|      WIN(W1=80AE%W2=8017%W3=802D%W4=80AE%W5=802F%W6=FFF7)
|_     T1(R=Y%DF=Y%TG=40%S=O%A=S+%F=AS%RD=0%Q=)
53/tcp   filtered domain     no-response
80/tcp   open     http       syn-ack
|  os: D-Link DWL-624+ or TRENDnet TEW-432BRP wireless broadband router (93%), Linksys BEFSR41 EtherFast broadband router or D-Link DCS-6620G webcam (93%), HP 9100c Digital Sender scanner (93%)
|      seq info: got packets=6 try=1/1 variance=2.828427
|      ECN(R=Y%DF=N%TG=FF%W=2000%O=M5B4%CC=N%Q=)
|      T2(R=N)
|      T3(R=N)
|      T4(R=N)
|      SEQ(SP=108%GCD=1%ISR=10E%TI=RD%TS=U)
|      OPS(O1=M5B4%O2=M5B4%O3=M5B4%O4=M5B4%O5=M5B4%O6=M5B4)
|      WIN(W1=2000%W2=2000%W3=2000%W4=2000%W5=2000%W6=2000)
|_     T1(R=Y%DF=N%TG=FF%S=O%A=S+%F=AS%RD=0%Q=)
113/tcp  filtered auth       no-response
443/tcp  open     https      syn-ack
|  os: D-Link DWL-624+ or TRENDnet TEW-432BRP wireless broadband router (93%), Linksys BEFSR41 EtherFast broadband router or D-Link DCS-6620G webcam (93%), HP 9100c Digital Sender scanner (93%)
|      seq info: got packets=6 try=1/1 variance=1.154701
|      ECN(R=Y%DF=N%TG=FF%W=2000%O=M5B4%CC=N%Q=)
|      T2(R=N)
|      T3(R=N)
|      T4(R=N)
|      SEQ(SP=105%GCD=1%ISR=106%TI=RD%TS=U)
|      OPS(O1=M5B4%O2=M5B4%O3=M5B4%O4=M5B4%O5=M5B4%O6=M5B4)
|      WIN(W1=2000%W2=2000%W3=2000%W4=2000%W5=2000%W6=2000)
|_     T1(R=Y%DF=N%TG=FF%S=O%A=S+%F=AS%RD=0%Q=)

I should also make it clear that p0f.nse don't sends any crafted packets (it only openes connection using standard connect()). But os.nse sends at least 10 crafted packet to every open port. That's not very stealthy method.


The sources are in my svn:

svn co svn://svn.insecure.org/nmap-exp/majek04/nmap-6861 nmap-majek

NSE loop bug

Nmap Scripting Engine is an engine for running Lua scripts inside Nmap. It's based on Nmap asynchronous library NSock.

During work on NSE script I found that sometimes Lua threads were frozen for too long. For example socket_object:receive() should take few milliseconds, but it took more than a hundred. In normal usage I wouldn't even notice that events are delayed, but this time I needed exact timings.

Like every asynchronous library NSock has special event loop. The main loop in NSE looks similar:

int process_mainloop(){
    ...
    while(unfinished_lua_threads > 0){
        ...
        nsock_loop(50ms); // block for at most 50ms, or till the first event
        ...
        if(running_scripts.begin() == running_scripts.end())
            continue;

        current = *(running_scripts.begin());
        // execute thread. It should be yielded back to waiting_scripts or end.
        lua_resume(current); 
        ...
    }
}

It's not obvious to spot the bug. The problem is that we handle only one Lua thread for one loop iteration. But it's possible that many events have occurred during nsock_loop.

This kind of bugs is quite common. For example PGBouncer recently had(still has?) similar bug while accepting connections. It allowed only one new connection per one event loop iteration.

The fixed loop will look like this:

int process_mainloop(){
    ...
    while(unfinished_lua_threads > 0){
        ...
        nsock_loop(50ms); // block for at most 50ms, or till the first event
        ...
        while(running_scripts.begin() != running_scripts.end()){
            ...
            current = *(running_scripts.begin());
            // execute thread. It should be yielded back to waiting_scripts or end.
            lua_resume(current); 
            ...
        }
    }
}

UPDATE #1 Fix is commited. It is in nmap svn version newer than r6857.

Nmap NSE-pcap example

At nmap-dev mailing list Lionel asked me about NSE-pcap. He wanted to send raw UDP packet and capture the response.

I created an example script that does this.
The patch against nmap.

Or if you prefer already patched files:

• packet.lua
• pcap-example.lua

Trinity choice: nsock over libevent

Everybody knows that Trinity is using nmap.
But what she has to nsock or libevent, well nothing.

Recently I wrote some software using libevent and I'm disgusted. Libevent is poorly documented (except the self-explanatory function names which are suggestive), there are not many examples on the web. Even though I took an effort and wrote few programs.

About a year ago I put my hands on nmap project, I wrote something for nmap's asynchronous networking library nsock.

There are some really interesting differences between these two libraries:

• In libevent you must allocate memory and set event structures by hand, from manual: "It is the responsibility of the caller to provide these functions with pre-allocated event structures.". In nsock event objects are managed by library.
• I thought that EV_PERSIST events should be persistent. I don't get exactly the idea of EV_PERSIST event type in libevent. It's not working for me when I think it should (for example for EV_WRITE). On the other hand nsock doesn't provide any kind of persistent events.
• Libevent provides the api for queuing signals. I haven't got an idea what application could theoretically take a use of such feature. I can't think of any use case for queuing signals in real world applications.
• Nsock executes callback function with the information about a state of an event like if it has succeeded, failed or timeouted. Libevent doesn't. User have to check the socket status by hand.
• In Nsock user can get current time (the time after last select() in main event loop) without syscall using nsock_gettimeofday(), which can speedup program. In libevent you have to use gettimefday().
• Libevent supports multiple polling methods, most notably epoll(), while nsock is using only slow select().
• Libevent could be used for both client or server side. Nsock is build especially for client side, it's possible to abuse it and use it for server side, but it's more like hacking.

The thing I like most in nsock is the easiness of using it. I don't have to worry about event structures. I have the guarantee that sooner or later my callback function will be called. To show the difference I have an example. It's the simplest program I could think of. It registers timer callback with null timeout, million times one after another sequentially.

First libevent example:

#include <stdlib.h>
#include <event.h>

 /* 
 gcc -Wall event_test1.c -levent && time ./a.out
 */

int counter = 0;
struct timeval tv = {0,0};
struct event ev;

void timer_handler(int fd, short event, void *ud){
    event_add(&ev, &tv);
    counter++;
    if(counter > 1000000)
        exit(0);
}

int main(){
    event_init();
    
    event_set(&ev, -1, EV_TIMEOUT, timer_handler, NULL);
    event_add(&ev, &tv);
    event_loop(0);
    return(0);
}

And very similar nsock code:

#include <nsock.h>
#include <stdlib.h>
 
 /* Sorry, some magic with compilation. Nsock isn't standalone library yet! 
 gcc -Wall nsock_test.c -I/home/majek/nmap/nsock/include /home/majek/nmap/nsock/src/libnsock.a -ldnet /home/majek/nmap/nbase/libnbase.a /home/majek/nmap/libpcap/libpcap.a && time ./a.out
 */

nsock_pool nsp; /* global */
int counter = 0;

void timer_handler (nsock_pool nsp, nsock_event nse, void *ud){
    nsock_timer_create(nsp, timer_handler, 0 /*ms*/, NULL);
    counter++;
    if(counter > 1000000)
        exit(0);
}

int main(){
    nsp = nsp_new(NULL);
    nsock_timer_create(nsp, timer_handler, 0 /*ms*/, NULL);
    nsock_loop(nsp, 1000000);
    return(0);
}

Times of execution this programs are very interesting for me. Unbelievably it seems that nsock is more than two times faster!
Libevent averate timings:

real    0m2.271s
user    0m0.640s
sys     0m1.624s

Nsock average time:

real    0m0.941s
user    0m0.392s
sys     0m0.548s

The "user time" shows that libevent implementation of internal structures is much heavier than nsock. Even that nsock should have more work because it's allocating event structures alone, without users help. The "sys time" should be similar for both libraries, but it's not. Strace is going to reveal the issue:

# The loop is reduced to 100 iterations.

# Data for NSOCK
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
   nan    0.000000           0       203           gettimeofday

# Data for LIBEVENT
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
   nan    0.000000           0       406           gettimeofday
   nan    0.000000           0       101           epoll_wait
   nan    0.000000           0         1           epoll_create
   nan    0.000000           0         1           epoll_ctl

Well, it seems clear that two things are broken in libevent. First, the gettimeofday() is called four times for every event loop iteration, it's unacceptable waste of resources. Second, the epoll() is executed for every event loop, while for nsock even a single select() isn't called. In this particular case, when timer events are registered with Null timeout there isn't a need of executing epoll().

Summary:
From programmers point of view, I like nsock programming style. Unfortunately nsock doesn't have many important features, like epoll or server side sockets handling.

It seems that libevent ain't no perfect either.

Maybe there is a need of async library with easy api like nsock and full-featured like libevent?

Nmap nse descriptors overflow patch

I just released quick patch for Brandon bug. It's interesting that this flaw wasn't found earlier.

My patch is a bit hackish, but it should work. I wonder if anyone finds better way of fixing this issue. The descriptor overflow patch lies here.

Update 1: Hurray! Patch is included in svn version of nmap (r5739).

Peek at new nmap-nse scripts

Well. I'll give you a peek at my new scripts for nmap. This scripts aren't public yet, I hope this post will give mi motivation to finish them.

This time we're going to focus on traceroute. New traceroute function in nmap looks like this:

# ./nmap -n -sS -p80 scanme.insecure.org --traceroute

TRACEROUTE (using port 80/tcp)
HOP RTT    ADDRESS
1   0.33   (censor)
2   7.93   (censor)
3   ...
4   7.84   212.76.35.50
5   23.85  212.76.35.58
6   4.90   212.76.35.177
7   5.06   217.6.51.49
8   209.72 62.154.5.9
9   191.55 64.125.12.169
10  196.77 64.125.26.26
11  198.69 64.125.28.142
12  200.40 208.185.168.173
13  200.15 205.217.153.62

My first script gives similar results. It works almost like standard traceroute, but instead of sending Syn packets with small ttl, it injects packets to established connection. The idea is stolen from Lcamtuf's 0trace tool.

# ./nmap -n -sS -p80 --script=0trace.nse scanme.insecure.org -P0
Interesting ports on 205.217.153.62:
PORT   STATE SERVICE
80/tcp open  http
|_ 0trace: 
(censor) 
(censor)
? 
212.76.35.50 
212.76.35.58 
212.76.35.177
217.6.51.49 
62.154.5.9 
64.125.12.169 
64.125.26.26 
64.125.28.142 
208.185.168.173 
205.217.153.62

Next script is quite different. It sends Syn packet with Record Route ip option (see ping -R). Note that this time ip addresses are different than in previous scans. That's because routers have several ip addresses and Record Route records ip from outgoing interface (I think). The disadvantage of this method is that it's possible to record only nine hops.

# ./nmap -n -sS -p80 --script=recordroute.nse scanme.insecure.org -P0
Interesting ports on 205.217.153.62:
PORT   STATE SERVICE
80/tcp open  http
|_ record route:
(censor)
(censor)
212.76.35.49
212.76.35.57
212.76.35.178
217.6.51.42
62.154.5.254
64.125.12.170
209.249.254.29

Some random Nmap ideas

There were some interesting projects that were using nmap. For example Doug Hoyte Qscan idea. I hope his work won't be forgotten.

But there are also some interesting projects on my desk.

I think there is a possibility to implement Lcamtuf 0trace as nse script. This could be an add on to my p0f script. In 0trace we're of course interested in last hops only because standard traceroute was already implemented in nmap by Eddie.

Adding bind(2) to nsock library can also be very interesting. The effect won't be tremendous, but implementing this seems really challenging.

The biggest project I would like to end writing is General Scanning Engine. Unfortunately the project needs to be completely rewritten. I would like to see GSE plugins written in Nmap Scripting Engine. I must think if there's simpler design than the design I proposed during Google's Summer of Code'06.

Pcap support for Nmap Script Engine

Some time ago I wrote nse-pcap patch for Nmap that adds some libpcap features to Diman's NSE. Today I ported changes to current Nmap. It's definitely time to check if the code is worth time I spent on it!

I think that pcap support one of the most promising features in NSE. It gives "new experience" to advanced programmers. It also introduces easy to use and powerful way of distributing received packets to registered Lua NSE threads. I think it is very nice way of programming pcap applications. Basically it's based on callbacks from pcap, rather than iterating over sequence of pcap results.

Good example of nse-pcap power can give my implemetation of Lcamtuf's p0f SYN+ACK scan.

Try out installation of nse-pcap.

$ svn co --username=guest --password= svn://svn.insecure.org/nmap-exp/soc07/nmap nmap
$ cd nmap
$ wget http://ai.pjwstk.edu.pl/~majek/private/nmap/nse-pcap/nmap-soc07-5184-ncap-try2B2-with-whitespace.diff
$ cat nmap-soc07-5184-ncap-try2B2-with-whitespace.diff|patch -p1
$ ./configure --without-nmapfe && make

And play with the power of p0f.nse script:

$ sudo ./nmap -n -sT -PS80 -p21,22,53,80,443 --script=p0f.nse www.cisco.com

Starting Nmap 4.22SOC1 ( http://insecure.org ) at 2007-07-12 00:04 CEST
Interesting ports on 198.133.219.25:
PORT    STATE    SERVICE
21/tcp  open     ftp
|_ p0f signature: UNKNOWN [65500:61:1:64:M1436,N,N,S,N,W0,N,N,T:AT:?:?] (link: IPSec/GRE, up: 1446 hrs, ipid:54702)
22/tcp  filtered ssh
53/tcp  filtered domain
80/tcp  open     http
|_ p0f signature: UNKNOWN [8192:238:0:44:M1460:A:?:?] (link: ethernet/modem, up: disabled, fill:4008, ipid:1)
443/tcp open     https
|_ p0f signature: UNKNOWN [8192:238:0:44:M1460:A:?:?] (link: ethernet/modem, up: disabled, fill:f8ef, ipid:10)

From this information I can see at least source NAT. And next try:

$ sudo ./nmap -n -sT -p21,22,53,80,443 --script=p0f.nse www.orkut.com

Starting Nmap 4.22SOC1 ( http://insecure.org ) at 2007-07-12 00:04 CEST
Interesting ports on 209.85.141.85:
PORT    STATE    SERVICE
21/tcp  open     ftp
|_ p0f signature: UNKNOWN [65500:61:1:64:M1436,N,N,S,N,W0,N,N,T:AT:?:?] (link: IPSec/GRE, up: 7282 hrs, ipid:30587)
22/tcp  filtered ssh
53/tcp  filtered domain
80/tcp  open     http
|_ p0f signature: UNKNOWN [8190:235:0:44:M1400:A:?:?] (link: sometimes DSL (2), up: disabled, fill:0f2d, ipid:28096)
443/tcp closed   https

I'm waiting for feedback or bug reports!

UPDATE #1:
Okay. I'm so enthusiastic about this project, because I wrote it. But there also some other promising features in nmap, like Swen's Web application detection.

UPDATE #2:
Some usefull links:

• Initial report with promiscuous.nse script


• p0f implementation


• current, updated release of nse-pcap path

Nmap na OLPC?

Wygląda na to że Fyodor chce odpalić nmapa na laptopach za $100. Ciekawe czy to się uda.

Trzeba będzie pewnie trochę poprawić obsługę wifi, zmniejszyć zużycie ramu...

Nmap nse-pcap, próba #2

Właśnie skończyłem testować patch do nmapa, o roboczej nazwie nse-pcap.

Zamiast opisywać jego działanie powiem tylko, że jest po prostu zajebisty.

Wkrótce postaram się opisać jego możliwości.