What happened to tcp flag URGENT, MSG_OOB and SIGURG?

Nobody today uses tcp urgent mode, so it's good topic to make some research on.

Usually when socket receives tcp packet with URG flag it treats it as normal tcp data. recv() is going to read urgent data as it was normal tcp stream. The only difference is that the last byte of data is discarded. The last byte in urgent data was always a problem due to incoherent rfc.

Pseudocode for this case:

server: send("ab", MSG_OOB)
client: recv() -> "a"

Tcp urgent packets whould trigger SIGURG for process that's listening on socket. It doesn't work until we set process pid of socket owner (thanks Michał for this tip).

if (fcntl(sd, F_SETOWN, getpid()) < 0) {
    perror("fcntl()");
    exit(-1);
}

The interesting thing is that after we enabled SIGURG it's possible to send a signal to remote process without sending any valid data. Such situation occures when process receives tcp URG packet with one byte payload. The only byte is discarded, no data is waiting on a socket, but SIGURG is sent to a process.

client: signal(SIGURG, handler);
client: fcntl(sd, F_SETOWN, getpid());
client: poll([sd], 1, 1000);
server: send("a", MSG_OOB);
client: signal SIGURG is received by a handler
client: poll ends, because of received signal
client: but there aren't any data waiting on a socket

That's all. Feel free to use draft source code of client and server I used for testing.

Peek at new nmap-nse scripts

Well. I'll give you a peek at my new scripts for nmap. This scripts aren't public yet, I hope this post will give mi motivation to finish them.

This time we're going to focus on traceroute. New traceroute function in nmap looks like this:

# ./nmap -n -sS -p80 scanme.insecure.org --traceroute

TRACEROUTE (using port 80/tcp)
HOP RTT    ADDRESS
1   0.33   (censor)
2   7.93   (censor)
3   ...
4   7.84   212.76.35.50
5   23.85  212.76.35.58
6   4.90   212.76.35.177
7   5.06   217.6.51.49
8   209.72 62.154.5.9
9   191.55 64.125.12.169
10  196.77 64.125.26.26
11  198.69 64.125.28.142
12  200.40 208.185.168.173
13  200.15 205.217.153.62

My first script gives similar results. It works almost like standard traceroute, but instead of sending Syn packets with small ttl, it injects packets to established connection. The idea is stolen from Lcamtuf's 0trace tool.

# ./nmap -n -sS -p80 --script=0trace.nse scanme.insecure.org -P0
Interesting ports on 205.217.153.62:
PORT   STATE SERVICE
80/tcp open  http
|_ 0trace: 
(censor) 
(censor)
? 
212.76.35.50 
212.76.35.58 
212.76.35.177
217.6.51.49 
62.154.5.9 
64.125.12.169 
64.125.26.26 
64.125.28.142 
208.185.168.173 
205.217.153.62

Next script is quite different. It sends Syn packet with Record Route ip option (see ping -R). Note that this time ip addresses are different than in previous scans. That's because routers have several ip addresses and Record Route records ip from outgoing interface (I think). The disadvantage of this method is that it's possible to record only nine hops.

# ./nmap -n -sS -p80 --script=recordroute.nse scanme.insecure.org -P0
Interesting ports on 205.217.153.62:
PORT   STATE SERVICE
80/tcp open  http
|_ record route:
(censor)
(censor)
212.76.35.49
212.76.35.57
212.76.35.178
217.6.51.42
62.154.5.254
64.125.12.170
209.249.254.29

Is it possible to abuse icmp?

I wonder what's going to happen when malicious user will inject crafted icmp packet with some error information (like Port Unreachable) to tcp connection. Will the connection be closed? You may think that to inject something like this it's needed to guess sequential numbers. That's not exactly correct. Icmp payload can be relatively small, specification says it has to carry at least 8 bytes from OSI 3th layer, so only tcp source and destination port must be included.

I can assume that you're going to use google. To block your connection I have to only send 65K packets with payload: from google, to you, from 80port to, and sequentially select your local port. I should guess after while. That's even less than 65K possibilities, on standard linux there are less than 32K possible local ports (see proc parameter ip_local_port_range).

The question is how common tcp stacks are interpreting such crafted Icmp message.

There are also other possibly vulnerable applications. For example most dns servers, to make recursive queries use source port 53 (see bind 'query-source address' option). How dns servers react for icmp errors with relatively small payload? Is it possible to perform Dos attack using this method?

UPDATE #1:
Well, of course port number in tcp header uses only 16 bits. So to 8 bytes in icmp payload should fit: source, destination port, seq number.

I change my question. How common tcp stacks interpret icmp packets with too short payload?

UPDATE #2:
Linux seems to be immune for this kind of attack. The function that parses icmp errors for tcp transmission is in linux/net/tcp_ipv4.c:tcp_v4_err(). The code says, that payload must be at least 8 bytes long. Source and destination port in the payload must be correct and sequence number must be in correct range. In other case icmp packet is simply dropped.

Some random Nmap ideas

There were some interesting projects that were using nmap. For example Doug Hoyte Qscan idea. I hope his work won't be forgotten.

But there are also some interesting projects on my desk.

I think there is a possibility to implement Lcamtuf 0trace as nse script. This could be an add on to my p0f script. In 0trace we're of course interested in last hops only because standard traceroute was already implemented in nmap by Eddie.

Adding bind(2) to nsock library can also be very interesting. The effect won't be tremendous, but implementing this seems really challenging.

The biggest project I would like to end writing is General Scanning Engine. Unfortunately the project needs to be completely rewritten. I would like to see GSE plugins written in Nmap Scripting Engine. I must think if there's simpler design than the design I proposed during Google's Summer of Code'06.

Pcap support for Nmap Script Engine

Some time ago I wrote nse-pcap patch for Nmap that adds some libpcap features to Diman's NSE. Today I ported changes to current Nmap. It's definitely time to check if the code is worth time I spent on it!

I think that pcap support one of the most promising features in NSE. It gives "new experience" to advanced programmers. It also introduces easy to use and powerful way of distributing received packets to registered Lua NSE threads. I think it is very nice way of programming pcap applications. Basically it's based on callbacks from pcap, rather than iterating over sequence of pcap results.

Good example of nse-pcap power can give my implemetation of Lcamtuf's p0f SYN+ACK scan.

Try out installation of nse-pcap.

$ svn co --username=guest --password= svn://svn.insecure.org/nmap-exp/soc07/nmap nmap
$ cd nmap
$ wget http://ai.pjwstk.edu.pl/~majek/private/nmap/nse-pcap/nmap-soc07-5184-ncap-try2B2-with-whitespace.diff
$ cat nmap-soc07-5184-ncap-try2B2-with-whitespace.diff|patch -p1
$ ./configure --without-nmapfe && make

And play with the power of p0f.nse script:

$ sudo ./nmap -n -sT -PS80 -p21,22,53,80,443 --script=p0f.nse www.cisco.com

Starting Nmap 4.22SOC1 ( http://insecure.org ) at 2007-07-12 00:04 CEST
Interesting ports on 198.133.219.25:
PORT    STATE    SERVICE
21/tcp  open     ftp
|_ p0f signature: UNKNOWN [65500:61:1:64:M1436,N,N,S,N,W0,N,N,T:AT:?:?] (link: IPSec/GRE, up: 1446 hrs, ipid:54702)
22/tcp  filtered ssh
53/tcp  filtered domain
80/tcp  open     http
|_ p0f signature: UNKNOWN [8192:238:0:44:M1460:A:?:?] (link: ethernet/modem, up: disabled, fill:4008, ipid:1)
443/tcp open     https
|_ p0f signature: UNKNOWN [8192:238:0:44:M1460:A:?:?] (link: ethernet/modem, up: disabled, fill:f8ef, ipid:10)

From this information I can see at least source NAT. And next try:

$ sudo ./nmap -n -sT -p21,22,53,80,443 --script=p0f.nse www.orkut.com

Starting Nmap 4.22SOC1 ( http://insecure.org ) at 2007-07-12 00:04 CEST
Interesting ports on 209.85.141.85:
PORT    STATE    SERVICE
21/tcp  open     ftp
|_ p0f signature: UNKNOWN [65500:61:1:64:M1436,N,N,S,N,W0,N,N,T:AT:?:?] (link: IPSec/GRE, up: 7282 hrs, ipid:30587)
22/tcp  filtered ssh
53/tcp  filtered domain
80/tcp  open     http
|_ p0f signature: UNKNOWN [8190:235:0:44:M1400:A:?:?] (link: sometimes DSL (2), up: disabled, fill:0f2d, ipid:28096)
443/tcp closed   https

I'm waiting for feedback or bug reports!

UPDATE #1:
Okay. I'm so enthusiastic about this project, because I wrote it. But there also some other promising features in nmap, like Swen's Web application detection.

UPDATE #2:
Some usefull links:

• Initial report with promiscuous.nse script


• p0f implementation


• current, updated release of nse-pcap path

Filesystems in 21st century

At studies I was taught that there are three levels in theoretical filesystems:

1. File name is tightly bound to the volume where file is stored (eg: on Microsoft Windows the path contains drive name, like c:\windows).
2. File name that doesn't say anything about place. But specified directory can be mounted as disk (in unixes; you don't know on which disk file lies, but you know that every file in some directory is on one disk).
3. Where you can't differ physical location by the directory. For example you could have every file in specified directory on different disk. It would be quite nice to store small files on fast SCSI storage, and big ones on cheap ATA drive, wouldn't it? (as far as I know no vendor implements this)
   

But I wasn't said anything about distributed filesystems. Well, there are some levels of developement.

Let's imagine:
You have everything on one disk. You need more space. You add disks and join them in some kind of raid or LVM. But what if you can't add any more disks? You have to buy new computer and split files somehow on both computers.

The first level is:

1. You can say on which machine file lies just by looking at it's name
• The simplest method is to count hash(filename) % number_of_computers. The problem is that if you want to add new computer, you have to recount hashes for _all_ of files. In case some files must be moved to new node.
• Second method is to count hash for filename, but this time count the reminder of some arbitrary chosen number like 100 (ie: hash(filename) % 100). Each host has ranges of hashes that it serves. For example first host serves all files with hash from 0 to 50. Second from 51 to 99. If you need to add next computer to pool you need to split hash ranges. To do this you need only to recount hashes for all files but at only one node.
  
2. At some point recounting hashes is just too expensive. Nowadays copying disks can last tenths hours. Simpler way to maintain namespace is to use database which contains all the metadata.
   
   The database knows on which node specified file lies. This adds next level of complexity, because to access file you have to first ask the database. But on the other hand once written file is not going to be moved anywhere.
   Adding new disks is costless.
   
   
3. The next problem occurs when single database of metadata is too big. Consider having 60MLN of files. Every file metadata occupies row in the database. Roughly counting 60MLN of rows * let's say 64 bytes per row = 4GB of metadata. It would be nice to save seeks on the disk on database and store data in RAM. And what happens if such huge database goes down?
   Well, I'm not the one that believes in database replication.
   
   So what's the solution? Spread the metadata across the nodes.
   
   Google solution is to add next abstraction layer. They use single namespace database in their filesystem, but with hard requirement that the amount of metadata is relatively small. With such requirement it's possible to provide really fast fall back in case that master nameserver fails.
   
   Huge number of files can be served, but from next abstraction layer. They split the data and metadata into relatively small continous sets and distributed across nodes. The master knows which range of keys is served by which server.
   
   Let's assume we want to access file 'abc.txt'. First we must ask the master server, on which server this file lies. Master stores key ranges for every 'tablet' server.
   
   Next, we connect to 'tablet' server and ask for specified file. To resolve file position 'tablet' server uses in memory B-tree.
   
   Using this design we have to: ask master server, than tablet server. In background there can be some requests to underlaying distributed filesystem, but we have guarantee that all this requests are answered directly from memory. The only disk seek is from the tablet server that serves requested file.
   

I must admit I like Google solution. The more I think about it, the more I read about it, the more I think that's the way it should be.

Lsi Logic site sql procedure

Anonone's interested in sql procedure used on Lsi Logic main website?