Provide NICs information a-la ifconfig #376

giampaolo · 2014-05-23T15:09:42Z

From g.rodola on May 10, 2013 03:16:52

This is somewhat related to issue 250 and aims to provide a replacement for 
ifconfig command on UNIX.
Whereas issue 250 aims to provide something like this:

    >>> psutil.network_ifaces()
    {'lo': nic(up=True, duplex=0, speed=0), 'eth0': nic(up=True, duplex=2, 
speed=100)}

...here we want to provide the IP address(es) and netmask associated with a 
network interface similarly to ifconfig.

It seems that on most (all?) POSIX system we can use getifaddrs(3).
Here's a Linux example using ctypes: 
http://carnivore.it/2010/07/22/python_-_getifaddrs It prints:

    {'eth0': {2: [{'addr': '192.168.1.2', 'netmask': '255.255.255.0'}],
              10: [{'addr': 'fe80::92e6:baff:fe80:e90d',
                    'netmask': 'ffff:ffff:ffff:ffff::',
                    'scope': 2L}],
              17: [{'addr': '90:e6:ba:80:e9:0d'}]},
     'lo': {2: [{'addr': '127.0.0.1', 'netmask': '255.0.0.0'}],
            10: [{'addr': '::1',
                  'netmask': 'ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff'}],
            17: [{'addr': '00:00:00:00:00:00'}]}}


Here's a similar implementation for OSX / BSD: 
http://carnivore.it/2010/07/22/python_-_getifaddrs On FreeBSD it prints:

    [ifaddrs(name='usbus0', flags=65537L, family=18, address='', netmask=None),
     ifaddrs(name='em0', flags=34883L, family=18, address='00:50:56:28:ec:d8', 
netmask=None),
     ifaddrs(name='em0', flags=34883L, family=2, address='10.31.8.132', 
netmask='255.255.255.0'),
     ifaddrs(name='usbus1', flags=65537L, family=18, address='', netmask=None),
     ifaddrs(name='plip0', flags=34832L, family=18, address='', netmask=None),
     ifaddrs(name='lo0', flags=32841L, family=18, address='', netmask=None),
     ifaddrs(name='lo0', flags=32841L, family=28, address='::1', 
netmask='ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff'),
     ifaddrs(name='lo0', flags=32841L, family=28, address='fe80:5::1', 
netmask='ffff:ffff:ffff:ffff::'),
     ifaddrs(name='lo0', flags=32841L, family=2, address='127.0.0.1', 
netmask='255.0.0.0')]


In terms of final API it seems natural to provide a single function which 
provides all of these infos in one shot (NIC status up/down, speed, duplex plus 
all associated addresses provided by getifaddrs) but given that the internal 
implementation uses very different approaches I prefer to treat and develop the 
two functionalities separately for now.

Original issue: http://code.google.com/p/psutil/issues/detail?id=376

The text was updated successfully, but these errors were encountered:

giampaolo · 2014-05-23T15:09:44Z

From phi...@cloudera.com on May 13, 2013 16:47:54

For what it's worth, http://alastairs-place.net/projects/netifaces/ is a 
library with what looks like similar goals to yours.

giampaolo · 2014-05-23T15:09:45Z

From g.rodola on May 15, 2013 16:42:42

Linux implementeation committed as revision 259cf1979d3a .

giampaolo · 2014-05-23T15:09:45Z

From g.rodola on May 16, 2013 18:57:05

FreeBSD implementation committed in revision f1e8283cb6a4 .

Status: Started

giampaolo · 2014-05-23T15:09:46Z

From techtonik@gmail.com on December 15, 2013 04:43:03

I can't see how can I get a list of all available network interface ids?

I am looking into some call to return ["lo", "eth0"] list in one step.
The next I'd like to do is to get all IPs for each interface "by-id".

giampaolo · 2014-05-23T15:09:46Z

From techtonik@gmail.com on December 15, 2013 04:46:58

Probably a hack, but I found this:

$ cat /proc/net/dev
Inter-|   Receive                                                |  Transmit
 face |bytes    packets errs drop fifo frame compressed multicast|bytes    
packets errs drop fifo colls carrier compressed
    lo: 9108970   99862    0    0    0     0          0         0  9108970   
99862    0    0    0     0       0          0
  eth0: 350377139  290149    0    0    0     0          0         0 31993244  
236043    0    0    0     0       0          0

giampaolo · 2014-05-23T15:09:47Z

From techtonik@gmail.com on December 15, 2013 04:49:27

So, I an list those as:

    ifaces = []
    with open("/proc/net/dev", "rb") as netstat:
         for i, line in enumerate(netstat):
             if i < 2:  # skip first two header lines
                 continue
             parts = line.split(':')
             ifaces += [parts[0].strip()]
    print ifaces

giampaolo · 2014-05-23T15:09:48Z

From g.rodola on December 16, 2013 05:30:31

I started developing this feature in a separate branch so that's why it is not 
available yet (Windows, OSX and Solaris implementations are still missing).
If you want a list of all available NIC names you can already do that by using:

    >>> import psutil
    >>> list(psutil.net_io_counters(pernic=True))
    ['lo', 'wlan0', 'eth1']
    >>>

Obviously you won't get the associated IP addresses.

giampaolo · 2014-05-23T15:09:48Z

From techtonik@gmail.com on December 16, 2013 07:45:17

This interface is very hackish. =)

giampaolo · 2014-05-23T15:09:49Z

From g.rodola on December 16, 2013 07:51:01

The main goal of net_io_counters() is to provide IO stats, not NIC names, 
that's why  "list(psutil.net_io_counters(pernic=True))" looks hackish.

giampaolo · 2014-05-23T15:09:50Z

From techtonik@gmail.com on December 16, 2013 08:48:35

I found an easy way to list all NICs:

    $ ls  /sys/class/net
    eth0  lo

giampaolo · 2014-05-23T15:09:50Z

From techtonik@gmail.com on December 16, 2013 08:50:05

Or more pythonically:

    >>> import os
    >>> os.listdir('/sys/class/net')
    ['lo', 'eth0']

The only problem now is to get IPs for those.

giampaolo · 2014-05-23T15:09:51Z

From g.rodola on December 16, 2013 08:52:14

The purpose of this ticket is not to provide an API to retrieve NIC names.
We already have a function which can indirectly be used to do that and when 
this issue will be closed as fixed we'll have 2.
I don't think providing yet another (3) API for the same functionality is a good idea.
We'll just use psutil.net_ifaces().keys() and be done with it.

giampaolo · 2014-09-23T20:25:37Z

Update: as of now it seems this function is implemented for all UNIX flavors. The only missing platform is Windows.

>>> import psutil
>>> from pprint import pprint as pp
>>> pp(psutil.net_i
psutil.net_if_addrs(     psutil.net_io_counters(  
>>> pp(psutil.net_if_addrs())
{'docker0': [snic(family=2, address='172.17.42.1', netmask='255.255.0.0', broadcast='172.17.42.1'),
             snic(family=10, address='fe80::64f5:58ff:fef9:779a%docker0', netmask='ffff:ffff:ffff:ffff::', broadcast=None),
             snic(family=17, address='66:f5:58:f9:77:9a', netmask=None, broadcast='ff:ff:ff:ff:ff:ff')],
 'lo': [snic(family=2, address='127.0.0.1', netmask='255.0.0.0', broadcast='127.0.0.1'),
        snic(family=10, address='::1', netmask='ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff', broadcast=None),
        snic(family=17, address='00:00:00:00:00:00', netmask=None, broadcast='00:00:00:00:00:00')],
 'lxcbr0': [snic(family=2, address='10.0.3.1', netmask='255.255.255.0', broadcast='10.0.3.255'),
            snic(family=10, address='fe80::b44e:7fff:fea5:4874%lxcbr0', netmask='ffff:ffff:ffff:ffff::', broadcast=None),
            snic(family=17, address='b6:4e:7f:a5:48:74', netmask=None, broadcast='ff:ff:ff:ff:ff:ff')],
 'wlan0': [snic(family=2, address='192.168.0.10', netmask='255.255.255.0', broadcast='192.168.0.255'),
           snic(family=10, address='2a02:8109:83c0:224c::5', netmask='ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff', broadcast=None),
           snic(family=10, address='2a02:8109:83c0:224c:3975:9903:34cd:cc60', netmask='ffff:ffff:ffff:ffff::', broadcast=None),
           snic(family=10, address='2a02:8109:83c0:224c:c685:8ff:fe45:641', netmask='ffff:ffff:ffff:ffff::', broadcast=None),
           snic(family=10, address='fe80::c685:8ff:fe45:641%wlan0', netmask='ffff:ffff:ffff:ffff::', broadcast=None),
           snic(family=17, address='c4:85:08:45:06:41', netmask=None, broadcast='ff:ff:ff:ff:ff:ff')]}
>>>

Everything works fine on my laptop but on Travis we get a segfault:
https://travis-ci.org/giampaolo/psutil/jobs/36085270
This is a big showstopper as I cannot reproduce the issue.

I'm not sure about the function name yet (net_if_addrs) because of #250. We might have a single net_ifaces() function returning both the addresses associated with the interface and other information about it (namely speed, duplex and is_up).

giampaolo · 2014-09-24T19:58:38Z

Travis build fixed. I also exposed AF_LINK on OSX/ BSD for retrieving MAC addresses.
Open question: how to retrieve MAC addresses on all other UNIXes?
TODO: Windows implementation.
Current API:

>>> psutil.net_if_addrs()
{'lo': [snic(family=2, address='127.0.0.1', netmask='255.0.0.0', broadcast='127.0.0.1'),
        snic(family=10, address='::1', netmask='ffff:ffff:ffff:ffff', broadcast=None)],
 'wlan0': [snic(family=2, address='192.168.0.10', netmask='255.255.255.0', broadcast='192.168.0.255'),
           snic(family=10, address='2a02:8109:83c0:224c::5', netmask='ffff:ffff:ffff', broadcast=None)]}
>>>

…st addresses retrieval

giampaolo · 2015-02-06T19:03:23Z

Update:

MAC addresses are now returned on all UNIX flavors and also on Windows
on OSX and BSD a MAC address has family == AF_LINK
on Linux it's == AF_PACKET
Windows currently temporarily returns family == -1 because the socket module does not expose anything to identify a MAC address
AF_LINK is exposed as psutil.AF_LINK because it made its appearance in the socket module only as of Python 3.4 (see: http://bugs.python.org/issue17996)
on Windows we are currently able to get the IP address and the MAC addres but not the netmask and broadcast address. netifaces library is able to do that but the code is pretty convoluted. I suppose it's OK for now to come up with a new release which only returns IP and MAC addresses and try to implement netmask and broadcast addresses later
speaking of which... what about default gateway? (it only exists on Windows)
...and what about DNS? (again, it only exists on Windows)

Open questions:

should we provide a single psutil.HWADDR constant and while iterating over address change AF_LINK or AF_PACKET to psutil.HWADDR? (probably not)
this would help fixing issue number 4 for Windows though.
should the family be a socket enum on Pyhon >= 3.4 (as opposed to a plain integer)? This also applies for psutil.net_connections() and psutil.Process.open_connections()

Reminder: see what netifaces does in this regard.

giampaolo · 2015-02-06T22:40:53Z

netifaces provides an AF_LINK constants on all platforms: on Linux it's == socket.AF_PACKET, on FreeBSD it's == socket.AF_LINK, on Windows it's == -1000. I guess we'll do the same.
Side note: this means psutil.AF_LINK won't be portable across different platforms in case the data is serialized and sent to a remote host but we already have the same issue with all the other AF_* constants returned by psutil.net_connections() and psutli.Process.connections().

…rimary addresses which cannot be termined

giampaolo · 2015-02-09T14:27:56Z

Merged into master as #588.

giampaolo self-assigned this May 23, 2014

giampaolo closed this as completed May 26, 2014

giampaolo reopened this Sep 23, 2014

giampaolo added a commit that referenced this issue Oct 4, 2014

#376: implement ifconfig con Windows - still missing: MAC and broadca…

aabc21e

…st addresses retrieval

giampaolo added a commit that referenced this issue Oct 5, 2014

#376: MAC address on Windows

b1189a3

giampaolo added a commit that referenced this issue Feb 6, 2015

#376: make net_if_addr() work on all other UNIX variants; also skip p…

f54979e

…rimary addresses which cannot be termined

giampaolo added a commit that referenced this issue Feb 6, 2015

#376: add another test for verifying the returned NIC names

de9f48b

giampaolo added a commit that referenced this issue Feb 7, 2015

#376: unify AF_LINK constant across all platforms + update doc and tests

80b04a5

giampaolo mentioned this issue Feb 7, 2015

Turn *_connections() constants into enums #582

Closed

giampaolo closed this as completed Feb 9, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Provide NICs information a-la ifconfig #376

Provide NICs information a-la ifconfig #376

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented Sep 23, 2014

giampaolo commented Sep 24, 2014

giampaolo commented Feb 6, 2015

giampaolo commented Feb 6, 2015

giampaolo commented Feb 9, 2015

Provide NICs information a-la ifconfig #376

Provide NICs information a-la ifconfig #376

Comments

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented May 23, 2014

giampaolo commented Sep 23, 2014

giampaolo commented Sep 24, 2014

giampaolo commented Feb 6, 2015

giampaolo commented Feb 6, 2015

giampaolo commented Feb 9, 2015