how to audit for open services with iproute2
The computer world has a tendency of reinventing the wheel once in a
while. I am not a fan of that process, but sometimes I just have to
bite the bullet and adapt to change. This post explains how I adapted
to one particular change: the netstat to sockstat transition.
I used to do this to show which processes where listening on which port on a server:
netstat -anpe
It was a handy mnemonic as, in France, ANPE was the agency
responsible for the unemployed (basically). That would list all
sockets (-a), not resolve hostnames (-n, because it's slow), show
processes attached to the socket (-p) with extra info like the user
(-e). This still works, but sometimes fail to find the actual
process hooked to the port. Plus, it lists a whole bunch of UNIX
sockets and non-listening sockets, which are generally irrelevant
for such an audit.
What I really wanted to use was really something like:
netstat -pleunt | sort
... which has the "pleut" mnemonic ("rains", but plural, which makes
no sense and would be badly spelled anyway). That also only lists
listening (-l) and network sockets, specifically UDP (-u) and TCP
(-t).
But enough with the legacy, let's try the brave new world of sockstat
which has the unfortunate acronym ss.
The equivalent sockstat command to the above is:
ss -pleuntO
It's similar to the above, except we need the -O flag otherwise ss
does that confusing thing where it splits the output on multiple
lines. But I actually use:
ss -plunt0
... i.e. without the -e as the information it gives (cgroup, fd
number, etc) is not much more useful than what's already provided with
-p (service and UID).
All of the above also show sockets that are not actually a concern because they only listen on localhost. Those one should be filtered out. So now we embark into that wild filtering ride.
This is going to list all open sockets and show the port number and service:
ss -pluntO --no-header | sed 's/^\([a-z]*\) *[A-Z]* *[0-9]* [0-9]* *[0-9]* */\1/' | sed 's/^[^:]*:\(:\]:\)\?//;s/\([0-9]*\) *[^ ]*/\1\t/;s/,fd=[0-9]*//' | sort -gu
For example on my desktop, it looks like:
anarcat@angela:~$ sudo ss -pluntO --no-header | sed 's/^\([a-z]*\) *[A-Z]* *[0-9]* [0-9]* *[0-9]* */\1/' | sed 's/^[^:]*:\(:\]:\)\?//;s/\([0-9]*\) *[^ ]*/\1\t/;s/,fd=[0-9]*//' | sort -gu
[::]:* users:(("unbound",pid=1864))
22 users:(("sshd",pid=1830))
25 users:(("master",pid=3150))
53 users:(("unbound",pid=1864))
323 users:(("chronyd",pid=1876))
500 users:(("charon",pid=2817))
631 users:(("cups-browsed",pid=2744))
2628 users:(("dictd",pid=2825))
4001 users:(("emacs",pid=3578))
4500 users:(("charon",pid=2817))
5353 users:(("avahi-daemon",pid=1423))
6600 users:(("systemd",pid=3461))
8384 users:(("syncthing",pid=232169))
9050 users:(("tor",pid=2857))
21027 users:(("syncthing",pid=232169))
22000 users:(("syncthing",pid=232169))
33231 users:(("syncthing",pid=232169))
34953 users:(("syncthing",pid=232169))
35770 users:(("syncthing",pid=232169))
44944 users:(("syncthing",pid=232169))
47337 users:(("syncthing",pid=232169))
48903 users:(("mosh-client",pid=234126))
52774 users:(("syncthing",pid=232169))
52938 users:(("avahi-daemon",pid=1423))
54029 users:(("avahi-daemon",pid=1423))
anarcat@angela:~$
But that doesn't filter out the localhost stuff, lots of false positive (like emacs, above). And this is where it gets... not fun, as you need to match "localhost" but we don't resolve names, so you need to do some fancy pattern matching:
ss -pluntO --no-header | \
sed 's/^\([a-z]*\) *[A-Z]* *[0-9]* [0-9]* *[0-9]* */\1/;s/^tcp//;s/^udp//' | \
grep -v -e '^\[fe80::' -e '^127.0.0.1' -e '^\[::1\]' -e '^192\.' -e '^172\.' | \
sed 's/^[^:]*:\(:\]:\)\?//;s/\([0-9]*\) *[^ ]*/\1\t/;s/,fd=[0-9]*//' |\
sort -gu
This is kind of horrible, but it works, those are the actually open ports on my machine:
anarcat@angela:~$ sudo ss -pluntO --no-header | sed 's/^\([a-
z]*\) *[A-Z]* *[0-9]* [0-9]* *[0-9]* */\1/;s/^tcp//;s/^udp//' |
grep -v -e '^\[fe80::' -e '^127.0.0.1' -e '^\[::1\]' -e '^192\.' -
e '^172\.' | sed 's/^[^:]*:\(:\]:\)\?//;s/\([0-9]*\) *[^ ]*/\
1\t/;s/,fd=[0-9]*//' | sort -gu
22 users:(("sshd",pid=1830))
500 users:(("charon",pid=2817))
631 users:(("cups-browsed",pid=2744))
4500 users:(("charon",pid=2817))
5353 users:(("avahi-daemon",pid=1423))
6600 users:(("systemd",pid=3461))
21027 users:(("syncthing",pid=232169))
22000 users:(("syncthing",pid=232169))
34953 users:(("syncthing",pid=232169))
35770 users:(("syncthing",pid=232169))
48903 users:(("mosh-client",pid=234126))
52938 users:(("avahi-daemon",pid=1423))
54029 users:(("avahi-daemon",pid=1423))
Surely there must be a better way. It turns out that lsof can do
some of this, and it's relatively straightforward. This lists all
listening TCP sockets:
lsof -iTCP -sTCP:LISTEN +c 15 | grep -v localhost | sort
A shorter version from Adam Shand is:
lsof -i @localhost
... which basically replaces the grep -v localhost line.
In theory, this would do the equivalent on UDP
lsof -iUDP -sUDP:^Idle
... but in reality, it looks like lsof on Linux can't figure out the state of a UDP socket:
lsof: no UDP state names available: UDP:^Idle
... which, honestly, I'm baffled by. It's strange because ss can
figure out the state of those sockets, heck it's how -l vs -a
works after all. So we need something else to show listening UDP
sockets.
The following actually looks pretty good after all:
ss -pluO
That will list localhost sockets of course, so we can explicitly ask
ss to resolve those and filter them out with something like:
ss -plurO | grep -v localhost
oh, and look here! ss supports pattern matching, so we can actually
tell it to ignore localhost directly, which removes that horrible
sed line we used earlier:
ss -pluntO '! ( src = localhost )'
That actually gives a pretty readable output. One annoyance is we can't really modify the columns here, so we still need some god-awful sed hacking on top of that to get a cleaner output:
ss -nplutO '! ( src = localhost )' | \
sed 's/\(udp\|tcp\).*:\([0-9][0-9]*\)/\2\t\1\t/;s/\([0-9][0-9]*\t[udtcp]*\t\)[^u]*users:(("/\1/;s/".*//;s/.*Address:Port.*/Netid\tPort\tProcess/' | \
sort -nu
That looks horrible and is basically impossible to memorize. But it sure looks nice:
anarcat@angela:~$ sudo ss -nplutO '! ( src = localhost )' | sed 's/\(udp\|tcp\).*:\([0-9][0-9]*\)/\2\t\1\t/;s/\([0-9][0-9]*\t[udtcp]*\t\)[^u]*users:(("/\1/;s/".*//;s/.*Address:Port.*/Port\tNetid\tProcess/' | sort -nu
Port Netid Process
22 tcp sshd
500 udp charon
546 udp NetworkManager
631 udp cups-browsed
4500 udp charon
5353 udp avahi-daemon
6600 tcp systemd
21027 udp syncthing
22000 udp syncthing
34953 udp syncthing
35770 udp syncthing
48903 udp mosh-client
52938 udp avahi-daemon
54029 udp avahi-daemon
Better ideas welcome.
Update, one of the comments on mastodon suggested this:
ss -pluntO '! ( src = localhost )' | awk '{ printf("%-5s\t%-7s\t%-40s\t%s\n", $1, $2, $5, $6); }'
Which looks like this right now:
ss -pluntO '! ( src = localhost )' | awk '{ printf("%-5s\t%-7s\t%-40s\t%s\n", $1, $2, $5, $6); }'
Netid State Local Address:Port
udp UNCONN 192.168.122.1:53 0.0.0.0:*
udp UNCONN 0.0.0.0%virbr0:67 0.0.0.0:*
udp UNCONN 0.0.0.0:53478 0.0.0.0:*
udp UNCONN 0.0.0.0:53706 0.0.0.0:*
udp UNCONN 0.0.0.0:21027 0.0.0.0:*
udp UNCONN 0.0.0.0:53823 0.0.0.0:*
udp UNCONN 0.0.0.0:5353 0.0.0.0:*
udp UNCONN 0.0.0.0:5353 0.0.0.0:*
udp UNCONN 0.0.0.0:55250 0.0.0.0:*
udp UNCONN 0.0.0.0:38874 0.0.0.0:*
udp UNCONN 0.0.0.0:57321 0.0.0.0:*
udp UNCONN 0.0.0.0:41844 0.0.0.0:*
udp UNCONN [::]:45441 [::]:*
udp UNCONN [fe80::239a:8c0:9d46:2d08]%wlan0:546 [::]:*
udp UNCONN *:49909 *:*
udp UNCONN *:1716 *:*
udp UNCONN [::]:21027 [::]:*
udp UNCONN [::]:5353 [::]:*
udp UNCONN *:5353 *:*
udp UNCONN *:22000 *:*
udp UNCONN [::]:42853 [::]:*
tcp LISTEN 0.0.0.0:22 0.0.0.0:*
tcp LISTEN 192.168.122.1:53 0.0.0.0:*
tcp LISTEN [::]:22 [::]:*
tcp LISTEN *:1716 *:*
tcp LISTEN *:6600 *:*
tcp LISTEN *:22000 *:*
I'm not sure: it loses the process information, and adds a needless
Address:Port that's mostly null. But I like the awk idea, but would
strip this down to:
ss -pluntO '! ( src = localhost )' | awk '{ printf("%-5s\t%-40s\t%s\n", $1, $5, $7); }'
Which looks like this:
anarcat@angela:~> sudo ss -pluntO '! ( src = localhost )' | awk '{ printf("%-5s\t%-40s\t%s\n", $1, $5, $7); }'
Netid Local Peer
udp 192.168.122.1:53 users:(("dnsmasq",pid=4448,fd=5))
udp 0.0.0.0%virbr0:67 users:(("dnsmasq",pid=4448,fd=3))
udp 0.0.0.0:53478 users:(("syncthing",pid=6573,fd=18))
udp 0.0.0.0:21027 users:(("syncthing",pid=6573,fd=17))
udp 0.0.0.0:53823 users:(("mosh-client",pid=212545,fd=4))
udp 0.0.0.0:5353 users:(("avahi-daemon",pid=2541,fd=12))
udp 0.0.0.0:41844 users:(("avahi-daemon",pid=2541,fd=14))
udp [::]:45441 users:(("avahi-daemon",pid=2541,fd=15))
udp [fe80::239a:8c0:9d46:2d08]%wlan0:546 users:(("NetworkManager",pid=2693,fd=31))
udp [::]:21027 users:(("syncthing",pid=6573,fd=19))
udp [::]:5353 users:(("avahi-daemon",pid=2541,fd=13))
udp *:22000 users:(("syncthing",pid=6573,fd=14))
udp [::]:42853 users:(("syncthing",pid=6573,fd=16))
tcp 0.0.0.0:22 users:(("sshd",pid=2949,fd=7))
tcp 192.168.122.1:53 users:(("dnsmasq",pid=4448,fd=6))
tcp [::]:22 users:(("sshd",pid=2949,fd=6))
tcp *:6600 users:(("systemd",pid=6289,fd=67))
tcp *:22000 users:(("syncthing",pid=6573,fd=12))
Not bad! We still have a bunch of noise on the Local column because
we have a lot of *:, [::]: and 0.0.0.0: and lots of noise in the
Peer column because of that weird users: syntax.
But i think it still works pretty well.
One could add another sed in that pipeline to clear out at least the
Peer garbage like this:
ss -pluntO '! ( src = localhost )' | awk '{ printf("%-5s\t%-40s\t%s\n", $1, $5, $7); }' | sed 's/users:(("//;s/".*//'
... which then looks like:
anarcat@angela:~> sudo ss -pluntO '! ( src = localhost )' | awk '{ printf("%-5s\t%-40s\t%s\n", $1, $5, $7); }' | sed 's/users:(("//;s/".*//'
Netid Local Peer
udp 192.168.122.1:53 dnsmasq
udp 0.0.0.0%virbr0:67 dnsmasq
udp 0.0.0.0:53478 syncthing
udp 0.0.0.0:21027 syncthing
udp 0.0.0.0:53823 mosh-client
udp 0.0.0.0:5353 avahi-daemon
udp 0.0.0.0:41844 avahi-daemon
udp [::]:45441 avahi-daemon
udp [fe80::239a:8c0:9d46:2d08]%wlan0:546 NetworkManager
udp [::]:21027 syncthing
udp [::]:5353 avahi-daemon
udp *:22000 syncthing
udp [::]:42853 syncthing
tcp 0.0.0.0:22 sshd
tcp 192.168.122.1:53 dnsmasq
tcp [::]:22 sshd
tcp *:6600 systemd
tcp *:22000 syncthing
But we've sailed past the land of reasonable one-liner one can type by
heart there. I think our best bet is ss -pluntO '! ( src = localhost
)', if we can remember that...
Just remember to run ss as root otherwise it can't figure out who
owns all those sockets unless it's you.
You can use your Mastodon account to reply to this post.