NAME¶
lmc
—
device driver for LMC (now SBE) wide-area network
interface cards
SYNOPSIS¶
To wire this driver into your kernel, add the following line to your kernel
configuration file:
device lmc
Alternatively, to load this module at boot time, add
to
/boot/loader.conf; see
loader.conf(5).
To wire a line protocol into your kernel, add:
options NETGRAPH
device sppp
It is not necessary to wire line protocols into your kernel, they can be loaded
later with
kldload(8). The driver can send and
receive raw IP packets even if neither SPPP nor Netgraph are configured into
the kernel. Netgraph and SPPP can both be enabled; Netgraph will be used if
the
rawdata hook is connected.
DESCRIPTION¶
This is an open-source
UNIX device driver for PCI-bus
WAN interface cards. It sends and receives packets in HDLC frames over
synchronous circuits. A generic PC plus
UNIX plus some
LMC / SBE cards makes an
open router. This driver
works with
FreeBSD,
NetBSD,
OpenBSD,
BSD/OS and Linux OSs.
It has been tested on i386 (SMP 32-bit little-endian) and Sparc (64-bit
big-endian) architectures.
The
lmc
driver works with the following
cards:
- SBE wanADAPT-HSSI (LMC5200)
High Speed Serial Interface, EIA612/613, 50-pin connector, 0 to 52 Mb/s, DTE
only.
- SBE wanADAPT-T3 (LMC5245)
T3: two 75-ohm BNC connectors, C-Parity or M13 Framing, 44.736 Mb/s, up to
950 ft.
- SBE wanADAPT-SSI (LMC1000)
Synchronous Serial Interface, V.35, X.21, EIA449, EIA530(A), EIA232, 0 to 10
Mb/s, DTE or DCE.
- SBE wanADAPT-T1E1 (LMC1200)
T1 or E1: RJ45 conn, 100 or 120 ohms, T1-ESF-B8ZS, T1-SF-AMI,
E1-(many)-HDB3, 1.544 Mb/s or 2.048 Mb/s, up to 6 Kft.
Cards contain a high-performance
PCI interface, an
HDLC function and either integrated
modems (T1, T3) or
modem interfaces (HSSI and SSI).
- PCI
- The PCI interface is a DEC 21140A "Tulip" Fast Ethernet chip.
This chip has an efficient PCI implementation with scatter/gather DMA, and
can run at 100 Mb/s full duplex (twice as fast as needed here).
- HDLC
- The HDLC functions (ISO-3309: flags, bit-stuffing, CRC) are implemented in
a Field Programmable Gate Array (FPGA) which talks to the Ethernet chip
through a Media Independent Interface (MII). The hardware in the FPGA
translates between Ethernet packets and HDLC frames on-the-fly; think it
as a WAN PHY chip for Ethernet.
- Modem
- The modem chips are the main differences between cards. HSSI cards use
ECL10K chips to implement the EIA-612/613 interface. T3 cards use a
TranSwitch TXC-03401 framer chip. SSI cards use Linear Technology LTC1343
modem interface chips. T1 cards use a BrookTree/Conexant/Mindspeed Bt8370
framer and line interface chip.
Line protocols exist above device drivers and below internet protocols. They
typically encapsulate packets in HDLC frames and deal with higher-level issues
like protocol multiplexing and security. This driver is compatible with
several line protocol packages:
- Netgraph
- netgraph(4) implements many basic
packet-handling functions as kernel loadable modules. They can be
interconnected in a graph to implement many protocols. Configuration is
done from userland without rebuilding the kernel. Packets are sent and
received through this interface if the driver's
rawdata hook is connected, otherwise the
ifnet interface (SPPP and RawIP) is used. ASCII configuration control
messages are not currently supported.
- SPPP
- sppp(4) implements Synchronous-PPP,
Frame-Relay and Cisco-HDLC in the kernel.
- RawIP
- This null line protocol, built into the driver, sends and receives raw
IPv4 and IPv6 packets in HDLC frames (aka IP-in-HDLC) with no extra bytes
of overhead and no state at the end points.
EXAMPLES¶
ifconfig and lmcconfig¶
The program
lmcconfig(8) manipulates interface
parameters beyond the scope of
ifconfig(8). In
normal operation only a few arguments are needed:
-X
- selects the external SPPP line protocol package.
-x
- selects the built-in RawIP line protocol package.
-Z
- selects PPP line protocol.
-z
- selects Cisco-HDLC line protocol.
-F
- selects Frame-Relay line protocol.
lmcconfig
lmc0
- displays interface configuration and status.
lmcconfig
lmc0 -D
- enables debugging output from the device driver only.
ifconfig
lmc0 debug
- enables debugging output from the device driver and from the line protocol
module above it. Debugging messages that appear on the console are also
written to file /var/log/messages.
Caution: when things go very wrong, a torrent
of debugging messages can swamp the console and bring a machine to its
knees.
Operation¶
Activate a PPP link using SPPP and Netgraph with:
ngctl mkpeer lmc0: sppp rawdata downstream
ifconfig sppp0 10.0.0.1 10.0.0.2
Activate a PPP link using only SPPP with:
lmcconfig lmc0 -XYZ
ifconfig lmc0 10.0.0.1 10.0.0.2
Activate a Cisco-HDLC link using SPPP and Netgraph with:
ngctl mkpeer lmc0: sppp rawdata downstream
ifconfig sppp0 10.0.0.1 10.0.0.2 link2
Activate a Cisco-HDLC link using only SPPP with:
lmcconfig lmc0 -XYz
ifconfig lmc0 10.0.0.1 10.0.0.2
Activate a Cisco-HDLC link using only Netgraph with:
ngctl mkpeer lmc0: cisco rawdata downstream
ngctl mkpeer lmc0:rawdata iface inet inet
ifconfig ng0 10.0.0.1 10.0.0.2
Activate a Frame-Relay DTE link using SPPP with:
lmcconfig lmc0 -XYF
ifconfig lmc0 10.0.0.1 10.0.0.2
(SPPP implements the ANSI T1.617 annex D LMI.)
Activate a Frame-Relay DTE link using Netgraph with:
ngctl mkpeer lmc0: frame_relay rawdata downstream
ngctl mkpeer lmc0:rawdata lmi dlci0 auto0
ngctl connect lmc0:rawdata dlci0 dlci1023 auto1023
ngctl mkpeer lmc0:rawdata rfc1490 dlci500 downstream
ngctl mkpeer lmc0:rawdata.dlci500 iface inet inet
ifconfig ng0 10.0.0.1 10.0.0.2
This is
ONE possible Frame Relay configuration;
there are many.
Activate a RAWIP link using only the driver with:
lmcconfig lmc0 -x
ifconfig lmc0 10.0.0.1 10.0.0.2
Activate a RAWIP link using Netgraph with:
ngctl mkpeer lmc0: iface rawdata inet
ifconfig ng0 10.0.0.1 10.0.0.2
If the driver is unloaded and then loaded, reconnect hooks by:
ngctl connect lmc0: ng0: rawdata
inet
TESTING¶
Testing with Loopbacks¶
Testing with loopbacks requires only one card. Packets can be looped back at
many points: in the PCI chip, in the modem chips, through a loopback plug, in
the local external equipment, or at the far end of a circuit.
Activate the card with
ifconfig(8):
ifconfig lmc0 10.0.0.1
10.0.0.1
All cards can be looped through the PCI chip. Cards with internal modems can be
looped through the modem framer and the modem line interface. Cards for
external modems can be looped through the driver/receiver chips. See
lmcconfig(8) for details.
Loopback plugs test everything on the card.
- HSSI
- Loopback plugs can be ordered from SBE (and others). Transmit clock is
normally supplied by the external modem. When an HSSI card is operated
with a loopback plug, the PCI bus clock must be used as the transmit
clock, typically 33 MHz. When testing an HSSI card with a loopback plug,
configure it with lmcconfig(8):
lmcconfig lmc0 -a 2
“-a
2
” selects the PCI bus clock as the
transmit clock.
- T3
- Connect the two BNC jacks with a short coax cable.
- SSI
- Loopback plugs can be ordered from SBE (only). Transmit clock is normally
supplied by the external modem. When an SSI card is operated with a
loopback plug, the on-board clock synthesizer must be used. When testing
an SSI card with a loopback plug, configure it with
lmcconfig(8):
lmcconfig lmc0 -E -f
10000000
-E
puts the card in DCE mode to source a
transmit clock. “-f
10000000
” sets the internal clock source to
10 Mb/s.
- T1/E1
- A loopback plug is a modular plug with two wires connecting pin 1 to pin 4
and pin 2 to pin 5.
One can also test by connecting to a local modem (HSSI and SSI) or NI (T1 and
T3) configured to loop back. Cards can generate signals to loopback remote
equipment so that complete circuits can be tested; see
lmcconfig(8) for details.
Testing with a Modem¶
Testing with a modem requires two cards of different types.
- T3/HSSI
- If you have a T3 modem with an HSSI interface (made by Digital Link,
Larscom, Kentrox etc.) then use an HSSI card in one machine and a T3 card
in the other machine. The T3 coax cables must use the null modem
configuration (see below).
- T1/V.35
- If you have a T1 (or E1) modem with a V.35, X.21 or EIA530 interface, then
use an SSI card in one machine and a T1 card in the other machine. Use a
T1 null modem cable (see below).
Testing with a Null Modem Cable¶
Testing with a null modem cable requires two cards of the same type.
- HSSI
- Three-meter HSSI null-modem cables can be ordered from SBE. In a pinch, a
50-pin SCSI-II cable up to a few meters will work as a straight HSSI cable
(not a null modem cable). Longer cables should be purpose-built HSSI
cables because the cable impedance is different. Transmit clock is
normally supplied by the external modem. When an HSSI card is connected by
a null modem cable, the PCI bus clock can be used as the transmit clock,
typically 33 MHz. When testing an HSSI card with a null modem cable,
configure it with lmcconfig(8):
lmcconfig lmc0 -a 2
“-a
2
” selects the PCI bus clock as the
transmit clock.
- T3
- T3 null modem cables are just 75-ohm coax cables with BNC connectors. TX
OUT on one card should be connected to RX IN on the other card. In a
pinch, 50-ohm thin Ethernet cables usually
work up to a few meters, but they will not
work for longer runs — 75-ohm coax is
required.
- SSI
- Three-meter SSI null modem cables can be ordered from SBE. An SSI null
modem cable reports a cable type of V.36/EIA449. Transmit clock is
normally supplied by the external modem. When an SSI card is connected by
a null modem cable, an on-board clock synthesizer is used. When testing an
SSI card with a null modem cable, configure it with
lmcconfig(8):
lmcconfig lmc0 -E -f
10000000
-E
puts the card in DCE mode to source a
transmit clock. “-f
10000000
” sets the internal clock source to
10 Mb/s.
- T1/E1
- A T1 null modem cable has two twisted pairs that connect pins 1 and 2 on
one plug to pins 4 and 5 on the other plug. Looking into the cable entry
hole of a plug, with the locking tab oriented down, pin 1 is on the left.
A twisted pair Ethernet cable makes an excellent straight T1 cable. Alas,
Ethernet cross-over cables do not work as T1 null modem cables.
OPERATION NOTES¶
Packet Lengths¶
Maximum transmit and receive packet length is unlimited. Minimum transmit and
receive packet length is one byte.
Cleaning up after one packet and setting up for the next packet involves making
several DMA references. This can take longer than the duration of a short
packet, causing the adapter to fall behind. For typical PCI bus traffic levels
and memory system latencies, back-to-back packets longer than about 20 bytes
will always work (53 byte cells work), but a burst of several hundred
back-to-back packets shorter than 20 bytes will cause packets to be dropped.
This usually is not a problem since an IPv4 packet header is at least 20 bytes
long.
This device driver imposes no constraints on packet size. Most operating systems
set the default Maximum Transmission Unit (MTU) to 1500 bytes; the legal range
is usually (72..65535). This can be changed with
ifconfig lmc0 mtu 2000
SPPP enforces an MTU of (128..far-end-MRU) for PPP and 1500 bytes for
Cisco-HDLC. RAWIP sets the default MTU to 4032 bytes, but it can be changed to
anything.
BPF - Berkeley Packet Filter¶
This driver has hooks for
bpf(4), the Berkeley
Packet Filter. The line protocol header length reported to BPF is four bytes
for SPPP and P2P line protocols and zero bytes for RawIP.
To include BPF support into your kernel, add the following line to
conf/YOURKERNEL:
device bpf
To test the BPF kernel interface, bring up a link between two machines, then run
ping(8) and
tcpdump(1):
ping 10.0.0.1
and in a different window:
tcpdump -i lmc0
The output from
tcpdump(1) should look like this:
03:54:35.979965 10.0.0.2 > 10.0.0.1: icmp: echo request
03:54:35.981423 10.0.0.1 > 10.0.0.2: icmp: echo reply
Line protocol control packets will appear among the
ping(8) packets occasionally.
Device Polling¶
A T3 receiver can generate over 100K interrupts per second, this can cause a
system to “live-lock”: spend all of its time servicing
interrupts.
FreeBSD has a polling mechanism to prevent
live-lock.
FreeBSD's mechanism permanently disables interrupts from
the card and instead the card's interrupt service routine is called each time
the kernel is entered (syscall, timer interrupt, etc.) and from the kernel
idle loop; this adds some latency. The driver is permitted to process a
limited number of packets. The percentage of the CPU that can be consumed this
way is settable.
See the
polling(4) manpage for details on how to
enable the polling mode.
SNMP: Simple Network Management Protocol¶
This driver is aware of what is required to be a Network Interface Object
managed by an Agent of the Simple Network Management Protocol. The driver
exports SNMP-formatted configuration and status information sufficient for an
SNMP Agent to create MIBs for:
- RFC-2233: Interfaces group,
- RFC-2496: DS3 interfaces,
- RFC-2495: DS1/E1 interfaces,
- RFC-1659: RS232-like interfaces.
An SNMP Agent is a user program, not a kernel function. Agents can retrieve
configuration and status information by using Netgraph control messages or
ioctl(2) system calls. User programs should poll
sc->cfg.ticks which increments once per
second after the SNMP state has been updated.
HSSI and SSI LEDs¶
The card should be operational if all three green LEDs are on (the upper-left
one should be blinking) and the red LED is off. All four LEDs turn on at
power-on and module unload.
RED |
upper-right |
No Transmit clock |
GREEN |
upper-left |
Device driver is alive if blinking |
GREEN |
lower-right |
Modem signals are good |
GREEN |
lower-left |
Cable is plugged in (SSI only) |
T1E1 and T3 LEDs¶
The card should be operational if the upper-left green LED is blinking and all
other LEDs are off. For the T3 card, if other LEDs are on or blinking, try
swapping the coax cables! All four LEDs turn on at power-on and module unload.
RED |
upper-right |
Received signal is wrong |
GREEN |
upper-left |
Device driver is alive if blinking |
BLUE |
lower-right |
Alarm Information Signal (AIS) |
YELLOW |
lower-left |
Remote Alarm Indication (RAI) |
The green |
LED blinks if the device driver is alive. |
The red |
LED blinks if an outward loopback is active. |
The blue |
LED blinks if sending AIS, on solid if receiving AIS. |
The yellow |
LED blinks if sending RAI, on solid if receiving RAI. |
E1 Framing¶
Phone companies usually insist that customers put a
Frame Alignment Signal (FAS) in time slot 0. A
Cyclic Redundancy Checksum (CRC) can also ride in time slot 0.
Channel Associated Signalling (CAS) uses Time
Slot 16. In telco-speak
signalling is on/off
hook, ringing, busy, etc. Signalling is not needed here and consumes 64 Kb/s.
Only use E1-CAS formats if the other end insists on it! Use E1-FAS+CRC framing
format on a public circuit. Depending on the equipment installed in a private
circuit, it may be possible to use all 32 time slots for data (E1-NONE).
T3 Framing¶
M13 is a technique for multiplexing 28 T1s into a T3. Muxes use the C-bits for
speed-matching the tributaries. Muxing is not needed here and usurps the FEBE
and FEAC bits. Only use T3-M13 format if the other end insists on it! Use
T3-CParity framing format if possible. Loop Timing, Fractional T3, and HDLC
packets in the Facility Data Link are
not
supported.
T1 & T3 Frame Overhead Functions¶
- Performance Report Messages (PRMs) are enabled in T1-ESF.
- Bit Oriented Protocol (BOP) messages are enabled in T1-ESF.
- In-band loopback control (framed or not) is enabled in T1-SF.
- Far End Alarm and Control (FEAC) msgs are enabled in T3-CPar.
- Far End Block Error (FEBE) reports are enabled in T3-CPar.
- Remote Alarm Indication (RAI) is enabled in T3-Any.
- Loopbacks initiated remotely time out after 300 seconds.
T1/E1 'Fractional' 64 kb/s Time Slots¶
T1 uses time slots 24..1; E1 uses time slots 31..0. E1 uses TS0 for FAS overhead
and TS16 for CAS overhead. E1-NONE has
no
overhead, so all 32 TSs are available for data. Enable/disable time slots by
setting 32 1s/0s in a config param. Enabling an E1 overhead time slot, or
enabling TS0 or TS25-TS31 for T1, is ignored by the driver, which knows
better. The default TS param, 0xFFFFFFFF, enables the maximum number of time
slots for whatever frame format is selected. 56 Kb/s time slots are
not supported.
T1 Raw Mode¶
Special gate array microcode exists for the T1/E1 card. Each T1 frame of 24
bytes is treated as a packet. A raw T1 byte stream can be delivered to main
memory and transmitted from main memory. The T1 card adds or deletes framing
bits but does not touch the data. ATM cells can be transmitted and received
this way, with the software doing all the work. But that is not hard; after
all it is only 1.5 Mb/s second!
T3 Circuit Emulation Mode¶
Special gate array microcode exists for the T3 card. Each T3 frame of 595 bytes
is treated as a packet. A raw T3 signal can be
packetized, transported through a packet network
(using some protocol) and then
reconstituted as a
T3 signal at the far end. The output transmitter's bit rate can be controlled
from software so that it can be
frequency locked
to the distant input signal.
HSSI and SSI Transmit Clocks¶
Synchronous interfaces use two transmit clocks to eliminate
skew caused by speed-of-light delays in the modem
cable. DCEs (modems) drive ST, Send Timing, the first transmit clock. DTEs
(hosts) receive ST and use it to clock transmit data, TD, onto the modem
cable. DTEs also drive a copy of ST back towards the DCE and call it TT,
Transmit Timing, the second transmit clock. DCEs receive TT and TD and use TT
to clock TD into a flip flop. TT experiences the same delay as (and has no
skew relative to) TD. Thus, cable length does not
affect data/clock timing.
SEE ALSO¶
tcpdump(1),
ioctl(2),
bpf(4),
kld(4),
netgraph(4),
polling(4),
sppp(4),
loader.conf(5),
ifconfig(8),
lmcconfig(8),
mpd(8)
(
ports/net/mpd),
ngctl(8),
ping(8),
ifnet(9)
HISTORY¶
Ron Crane had the idea to use a Fast Ethernet
chip as a PCI interface and add an Ethernet-to-HDLC gate array to make a WAN
card.
David Boggs designed the
Ethernet-to-HDLC gate array and PC cards. We did this at our company, LAN
Media Corporation (LMC). SBE Corp. acquired LMC and continues to make the
cards.
Since the cards use Tulip Ethernet chips, we started with
Matt Thomas' ubiquitous
de(4) driver.
Michael
Graff stripped out the Ethernet stuff and added HSSI stuff.
Basil Gunn ported it to Solaris (lost) and
Rob Braun ported it to Linux.
Andrew
Stanley-Jones added support for three more cards and wrote the first
version of
lmcconfig(8).
David Boggs rewrote everything and now
feels responsible for it.
AUTHORS¶
David Boggs
⟨boggs@boggs.palo-alto.ca.us⟩