Feature request: allow the 802.1X NFV transport to share a port (VLAN-tagged or via packet-in/out) #4854

Summary

Currently nfv_sw_port must be a dedicated port with output_only: True. Configuring
a normal VLAN port as the NFV port is rejected with:

NFV Ports must have output_only set to True

This means an 802.1X deployment always needs an extra physical link (or veth pair) between
the switch and the Faucet host, dedicated to EAPOL. I'd like a way to carry the NFV traffic
over a port that is also used for other traffic, most usefully the controller's own port.

Use case

In-band control on a single link:

Router <--trunk--> OVS <--access (mgmt VLAN)--> Faucet host

• OVS reaches Faucet in-band over the management VLAN (hidden in-band flows).
• The Faucet host needs that same link for normal management traffic: reaching the
  router, a remote RADIUS server, radsecproxy, DNS, and so on.

Making that port output_only keeps the OpenFlow session up (the in-band flows take
priority), but it cuts the controller off from everything else, including a remote
RADIUS server. The only option is a second NIC or cable, which is awkward on small
appliances and in labs.

Proposed options

Option A: VLAN-tagged NFV transport (preferred, small change)

dot1x:
  nfv_intf: eth0.4094        # VLAN subinterface on the Faucet host
  nfv_sw_port: 2             # may now be a normal port (e.g. native_vlan: mgmt)
  nfv_vlan_vid: 4094         # new: tag EAPOL to/from the NFV port with this VID

• EAPOL to Chewie is output to nfv_sw_port with nfv_vlan_vid pushed, using the
  existing per-port encoded MACs.
• On ingress from nfv_sw_port, only frames tagged nfv_vlan_vid with the NFV encoded
  MACs are matched by the dot1x flows. Everything else on the port goes through the
  normal pipeline.
• Chewie is unchanged: it opens its raw socket on the host's VLAN subinterface and sees
  untagged frames as it does today.
• The output_only requirement applies only when nfv_vlan_vid is unset, so existing
  configs keep working.

Option B: packet-in/packet-out

Send EAPOL to the controller as a packet-in and inject replies as packet-outs, with no
NFV port at all. Faucet already handles ARP/ND for its VIPs and LLDP this way. This
removes the physical requirement entirely, but it's a bigger change, since Chewie
currently assumes a raw socket, and it adds packet-in load (which could be rate-limited
as with other packet-ins).

Prior art

• Before Chewie, Faucet's hostapd-based 802.1X sent EAPOL out a port with vlan_vid
  pushed via an ACL:
  https://github.com/proxy/gist.github.com/gizmoguy/085a13bbdb4aa61675c409c294385f74
• The faucet-dev list mentioned work on a coprocessor mechanism to send packets for
  different protocols to different coprocessors, with EAPOL to Chewie as an example.
  The current coprocessor port option (VLAN VID to port mapping) seems close to what
  Option A needs.

Related papercuts found while setting this up

Happy to split these into separate issues:

1. Missing guard for the 802.1X speaker. If a DP has a dot1x: block but no port
   has dot1x: True, Faucet crashes in nfv_sw_port_up with
   AttributeError: 'NoneType' object has no attribute 'port_down' (same as AttributeError: 'NoneType' object has no attribute 'port_down'  #3566).
   One way to hit this is setting dot1x: True only through interface_ranges for
   ports that aren't listed under interfaces:. Ranges don't create ports, so there
   are no dot1x ports. A config-check error ("dot1x block present but no dot1x ports")
   would be much clearer than the crash.
2. radius_port is documented with a default of 1812 but is actually required.
   Omitting it fails. Either apply the default or update the docs. Similar kind of crash
   with a Python exception.

Environment

• Faucet version: 1.10.12
• Chewie version: Built-in to Faucet 1.10.12
• Switch: Open vSwitch 2.17.12, in-band control, fail_mode=secure
• Running in Docker with network_mode: host.