FAULT TRACING DIRECT.     5/154 51-CRA 119 1170/1-V2 Uen C

BRAS Troubleshooting Guide 

SmartEdge OS Software

Copyright

© Ericsson AB 2009–2010. All rights reserved. No part of this document may be reproduced in any form without the written permission of the copyright owner.

Disclaimer

The contents of this document are subject to revision without notice due to continued progress in methodology, design and manufacturing. Ericsson shall have no liability for any error or damage of any kind resulting from the use of this document.

Trademark List

Contents

1   Getting to Know Your SmartEdge Router

This section shows how to display help and understand contexts, and also lists basic troubleshooting commands.

1.1   Getting Help for a Command

You can access the online Help for the command-line interface (CLI) in the following ways:

• Use the ? command when issuing a command to display the options available in the command syntax. The system provides two types: full help and partial help. Full help is available when you enter a command argument; for example, show ? describes each possible argument. Partial help is provided when an abbreviated argument is entered and you want to know what arguments match the input; for example, show pr?
• Use the help command to learn how to obtain help. Type the help command after the prompt.

Note:  

To include the ? character as part of a command when it is not used as a request for online Help, press the Esc key followed by the ? character.

1.2   Understanding Contexts

This section provides information about contexts and displaying debug output.

1.2.1   Understanding How the Active Context Affects Debug Output

Each context is an instance of a virtual SmartEdge® router that runs on the same physical device. A context operates as a separate routing-and-administrative domain with separate routing protocol instances, addressing, authentication, authorization, and accounting. A context does not share this information with other contexts.

There are two types of contexts: local (a system-wide context) and administrator defined (a nonlocal context). The active context (the context that you are in) affects your debug output.

Context-specific debugging refers to navigating to a specific context and running debug commands from it and filtering out all debug output not related to that context. The context-specific output are lines of output identified by a context ID in brackets, which can be displayed either using context-specific debugging or system-wide debugging.

1.2.1.1   Debugging from a Local Context

To debug all contexts on your router, use the system-wide local context. You see debug output related to this context and all contexts running on the router. For example, to see all OSPF instances on the router, issue the debug ospf lsdb command in the local context.

[local] Redback# debug ospf lsdb

When you debug a local context, the software displays debug output for all contexts. When a debug function is context specific, the debug output generated by the local context includes a context ID that you can use to determine the source of the event (the context in which the event has its origin). You can then navigate to the context that contains the event and collect additional information to troubleshoot it.

The following example displays debug output from a local context. The debug output generated by the local context using the show debug command includes a context ID 0005 (which is highlighted in bold). To find out the source of the debug event (the context name) for context ID 0005, issue the show context all command. In the Context ID column look for the Context ID with the last four digits is 0005—in this case, 0x40080005, which indicates that the source of the debug event is context Re-1.

Note:  

After a system reboot, context numbers might change.

Debug functions and the show context all command display context IDs in two different formats: decimal format and hexadecimal, respectively. For example, the debug output displays a context ID in decimal format as 0262; the show context all command displays the same ID in hexadecimal format as 0x40080106.

[local]Redback# show debug
OSPF:
   lsdb debugging is turned on
[local]Redback#
Apr 18 12:21:04: %LOG-6-SEC_STANDBY: Apr 18 12:21:04: %CSM-6-PORT:
ethernet 3/7 link state UP, admin is UP
Apr 18 12:21:04: %LOG-6-SEC_STANDBY: Apr 18 12:21:04: %CSM-6-PORT:
ethernet 3/8 link state UP, admin is UP
Apr 18 12:21:05: %CSM-6-PORT: ethernet 3/7 link state UP, admin is UP
Apr 18 12:21:05: %CSM-6-PORT: ethernet 3/8 link state UP, admin is UP
Apr 18 12:21:05: [0002]: %OSPF-7-LSDB: OSPF-1: Area 0.0.0.0 Update
Router LSA 200.1.1.1/200.1.1.1/80000013 cksum 26f1 len 72
Apr 18 12:21:05: [0003]: %OSPF-7-LSDB: OSPF-1: Area 0.0.0.2 
Update Router LSA 200.1.2.1/200.1.2.1/80000009 cksum ce79 len 36
Apr 18 12:21:05: [0004]: %OSPF-7-LSDB: OSPF-1: Area 0.0.0.3 
Update Sum-Net LSA 0.0.0.0/200.1.3.1/80000001 cksum bb74 len 28
Apr 18 12:21:05: [0004]: %OSPF-7-LSDB: OSPF-1: Area 0.0.0.3 
Update Router LSA 200.1.3.1/200.1.3.1/8000000a cksum 142 len 36
Apr 18 12:21:05: [0004]: %OSPF-7-LSDB: OSPF-1: Area 0.0.0.0 Update 
Router LSA 200.1.1.1/200.1.1.1/80000013 cksum 26f1 len 72
Apr 18 12:21:05: [0003]: %OSPF-7-LSDB: OSPF-1: Area 0.0.0.0 Update
Router LSA 200.1.1.1/200.1.1.1/80000013 cksum 26f1 len 72
Apr 18 12:21:06 [0005]: %OSPF-7-LSDB: OSPF-1: Area 0.0.0.0 Update 
// Associated with Context ID 0x40080005. This is context specific output,
in this case, context Re-1.


---------------------------------------------------------------- 

[local]Redback# show context all
Context Name      Context ID        VPN-RD         Description
-----------------------------------------------------------------
local              0x40080001
Rb-1               0x40080002
Rb-2               0x40080003
Rb-3               0x40080004
Re-1               0x40080005 // The source of the debug event for 
Re-2               0x40080006 // context ID 0005 is context Re-1.
Re-3               0x40080007
[local]Redback#

1.2.1.2   Debugging from a Specific Context

The SmartEdge router supports multiple contexts. The current context affects the output of some debug commands. For example, the debug ospf lsdb command can be context specific because multiple contexts can exist, each running its own protocols. In this example, you see only the OSPF debug output from context MyService. If you run the same command from the local context, you see output from all contexts that have OSPF enabled. The context ID in the debug message logs shows all the contexts for which this debug event is applicable. To debug a specific context for OSPF, navigate to that context—in this example, MyService.

[local]Redback# context MyService
[MyService] Redback# terminal monitor
[MyService] Redback# debug ospf lsdb 
OSPF:
lsdb debugging is turned on
[MyService]Redback#
Feb 27 15:11:24: [0001]: %OSPF-7-LSDB: OSPF-1: Area 0.0.0.0 Update
Router LSA 1.1.1.1/1.1.1.1/8000000c cksum ba60 len 36
Feb 27 15:11:24: [0001]: %OSPF-7-LSDB: OSPF-1: Delete 
Net:192.1.1.1[1.1.1.1] Area: 0.0.0.0
Feb 27 15:11:24: [0001]: %OSPF-7-LSDB: OSPF-1: Area 0.0.0.0 Update 
Router LSA 1.1.1.1/1.1.1.1/8000000d cksum b861 len 36
Feb 27 15:12:09: [0001]: %OSPF-7-LSDB: OSPF-1: Area 0.0.0.0 Update Net
LSA 192.1.1.1/1.1.1.1/80000002 cksum 1b4a len 32
Feb 27 15:12:09: [0001]: %OSPF-7-LSDB: OSPF-1: Delete 
Net:192.1.1.1[1.1.1.1] Area: 0.0.0.0
Feb 27 15:12:09: [0001]: %OSPF-7-LSDB: OSPF-1: Area 0.0.0.0 Update 
Router LSA 2.2.2.2/2.2.2.2/80000005 cksum 6ec8 len 36
Feb 27 15:12:09: [0001]: %OSPF-7-LSDB: OSPF-1: Area 0.0.0.0 Update 
Router LSA 1.1.1.1/1.1.1.1/80000010 cksum 4f30 len 48
Feb 27 15:12:09: [0001]: %OSPF-7-LSDB: OSPF-1: Area 0.0.0.0 Update Net
LSA 192.1.1.1/1.1.1.1/80000003 cksum 194b len 32
Feb 27 15:12:14: [0001]: %OSPF-7-LSDB: OSPF-1: Area 0.0.0.0 Update 
Router LSA 2.2.2.2/2.2.2.2/80000006 cksum 237a len 48

1.2.2   Identifying Context-Specific Debug Functions

Debug functions are either context specific or system wide. For example, the debug aaa authen command is system wide because negotiation takes place at the port or circuit level, and it is not associated with a context. When you debug from the local context, you see debug output from all contexts. If you debug a function from the local context, use the context ID to determine the source of the debug event (the context that the event is coming from). When you debug from a nonlocal context, you see output only from that context. You can perform context-specific debugging from the local context or from one of the contexts you have configured.

The following three examples show you how to recognize whether a debug function is context specific or applies to the local context. In the first example, context NiceService contains context identifier 0002, which indicates that the debug aaa author function is context specific.

The internal circuit handle, 13/1:1:63/1/2/11, consists of the following components:

1. slot/port—13/1
2. channel: subchannel—Identifies an individual circuit on a TDM port. 13/1:1:63 is an ATM circuit.
3. Authority (the application that made the circuit, in this case, ATM) 1, level of circuit (in this example, a traffic bearing Layer 2 circuit) 2, and the internal ID (a sequential uniquely assigned number) 11.

In the second example, the debug aaa authen function in the local context is system wide because no context identifiers are displayed in the show debug output. In the third example, the local context displays context identifiers, 0002, 0003, and 0004, which indicates that the source of the LSA updates are context specific. When you issue the show context all command, these contexts are displayed as 0x40080002, 0x40080003, and 0x40080004.

1.2.3   Displaying Debug Output through the Craft Port

Use the logging console command (in context configuration mode) to view event log messages on the console. By default, this is enabled in the local context.

[local]Redback#config
Enter configuration commands, one per line, 'end' to exit
[local]Redback(config)#context local
[local]Redback(config-ctx)#logging console

1.2.4   Displaying Debug Output Through Telnet or SSH

Use the terminal monitor command (in exec mode) to view event log messages on your terminal when you are connected through Telnet or SSH. To pause debug output at your terminal, type CTRL-S; to continue, type CTRL-C.

[local]Redback# terminal monitor

1.3   Debugging Tasks

Use Table 1 as a guide to troubleshooting subscriber connectivity issues. Check each task that you have completed and document your results. Before you begin, get a description of the problem from the customer and check if the customer has made any recent changes or upgrades to their network.

1.4   Using Basic Troubleshooting Commands

For more information about these commands, see the Command List.

2   Troubleshooting Subscriber Connectivity

This chapter shows you to troubleshoot subscriber connectivity problems. For information about troubleshooting general issues, including hardware, see the General Troubleshooting Guide.

Note:  

Some commands in this guide are not supported on versions prior to Release 6.2.1.

The following diagram shows the general procedure for troubleshooting subscriber software connectivity issues:

Use Table 3 as a guide to troubleshooting BRAS issues. Check each task that you have completed and document your results. Before you begin, get a description of the problem and check if you made any recent changes or upgrades to their network.

2.1   Step 1: Navigating to the Correct Context

Use the show context all command to view all your contexts and then navigate to the context that you want to troubleshoot. For information about contexts, see Understanding How the Active Context Affects Debug Output.

The following example shows how to view all contexts on your router and then navigate to context NiceService:

[local]Redback#show context all
Context Name        Context ID    VPN-RD   Description
------------------------------------------------------
local               0x40080001
NiceService         0x40080002
[local]Redback#
[local]Redback#context NiceService
[NiceService]Redback#

2.2   Step 2: Displaying Information About My Subscribers

Use the show subscribers command to display subscriber information within the current context. This includes basic subscriber status fields, DSL attributes of active subscriber sessions, Mobile IP attributes, AAA logs, a summary of subscriber information, and IP addresses associated with subscribers.

2.2.1   Displaying Active Subscribers

The show subscribers command displays subscriber usernames, circuits that they are associated with, the contexts that they are bound to. The active keyword provides information on the dynamic policy rules applied to active subscriber sessions.

The following illustration identifies the show subscribers active command output fields.

The following illustration identifies the show subscribers active command output fields for a DHCP lease.

The following example displays the information for an active subscriber; it includes both the absolute time-out action and traffic limit action fields:

[local]Redback# show subscribers active username client32@lns.com
client32@lns.com
Circuit   L2TP LNS 8744119
Internal Circuit   255/16:1023:63/5/2/8744119
Current port-limit unlimited
context-name lns (applied)
ip pool   (applied from sub_default)
absolute timeout action 1 (applied from sub_default)
traffic limit action 1 (applied from sub_default)
ip address 192.168.27.2 (applied from pool)
timeout absolute 60 (applied)
timeout idle 60 (applied)

Recommended Action: If you find a problem with the subscriber, use the debug circuit command and specify the circuit that you obtained from the show subscribers active username command to obtain more information about the issue.

The following example displays information about the DHCP hosts after they have been established on the active subscriber circuits:

[atm_subs]Redback#show subscribers active
sub1@atm_subs
        Circuit   1/4:1 vpi-vci 0 101
        Internal Circuit   1/4:1:63/1/2/24579
        Current port-limit unlimited
        profile gold (applied)
        dhcp max-addrs 10 (applied)
        ip interface gold (applied)
        IP host entries installed by DHCP: (max_addr 10 cur_enties 10)
                120.1.1.199    00:dd:00:00:00:0a
                120.1.1.191    00:dd:00:00:00:09
                120.1.1.192    00:dd:00:00:00:08
                120.1.1.200    00:dd:00:00:00:07
                120.1.1.194    00:dd:00:00:00:05
                120.1.1.193    00:dd:00:00:00:06
                120.1.1.196    00:dd:00:00:00:03
                120.1.1.195    00:dd:00:00:00:04
                120.1.1.197    00:dd:00:00:00:02
                120.1.1.198    00:dd:00:00:00:01
sub2@atm_subs
        Circuit   1/4:1 vpi-vci 0 102
        Internal Circuit   1/4:1:63/1/2/24580
        Current port-limit unlimited
        profile silver (applied)
        dhcp max-addrs 10 (applied)
        ip interface silver (applied)
          IP host entries installed by DHCP: (max_addr 10 cur_enties 10)
                120.1.2.191    00:dd:00:00:00:14
                120.1.2.192    00:dd:00:00:00:13
                120.1.2.193    00:dd:00:00:00:12
                120.1.2.194    00:dd:00:00:00:11
                120.1.2.195    00:dd:00:00:00:10
                120.1.2.196    00:dd:00:00:00:0f
                120.1.2.197    00:dd:00:00:00:0e
                120.1.2.198    00:dd:00:00:00:0d
                120.1.2.199    00:dd:00:00:00:0c
                120.1.2.200    00:dd:00:00:00:0b
sub3@atm_subs
        Circuit   1/4:1 vpi-vci 0 103
        Internal Circuit   1/4:1:63/1/2/24581
        Current port-limit unlimited
        profile bronze (applied)
        dhcp max-addrs 10 (applied)
        ip interface bronze (applied)
        IP host entries installed by DHCP: (max_addr 10 cur_enties 10)
                120.1.3.191    00:dd:00:00:00:1e
                120.1.3.192    00:dd:00:00:00:1d
                120.1.3.193    00:dd:00:00:00:1c
                120.1.3.194    00:dd:00:00:00:1b
                120.1.3.195    00:dd:00:00:00:1a
                120.1.3.196    00:dd:00:00:00:19
                120.1.3.197    00:dd:00:00:00:18
                120.1.3.198    00:dd:00:00:00:17
                120.1.3.199    00:dd:00:00:00:16
                120.1.3.200    00:dd:00:00:00:15

Use the show subscribers active handle command to find the circuit information (for example, useful for the (Slot/Port/PVC), when you only have the internal handle:

[local]Redback#show subscribers active handle 3/1:1023:63/1/2/12375 

Circuit 3/1 vpi-vci 0 380 =============>VPI-VCI Info.*
Internal Circuit 3/1:1023:63/1/2/12375*
Current port-limit unlimited
ip address 10.1.1.1 (applied)
dns primary 10.10.10.10 (not applied)
dns secondary 10.10.10.11 (not applied)

2.2.2   Displaying Information About All Subscribers

Use the show subscribers all command to display information about all subscribers in all contexts. The all keyword is available only to administrators in the local context. To clear a subscriber session, use the clear subscriber username command.

2.2.2.1   Displaying Information About All Subscribers in All Contexts

The following example shows how to display information about all subscribers in all contexts:

[local]Redback#show subscribers all

2.2.2.2   Displaying Information About an Individual Subscriber

The following example shows how to display information about the binding, context, and subscriber:

[local]Redback# show sub all | grep user2@NiceService 
pppoe 3/1 pppoe 7 user2@NiceService abc Aug 17 03:02:31

2.2.2.3   Displaying Summary Subscriber Information

The following example shows how to display summary information about subscribers:

[local]Redback# show subscribers summary  

2.2.3   Displaying Subscriber Logs

Use the show subscribers log command to display the authentication, authorization, and accounting (AAA) logs of subscribers.

2.2.3.1   Wrong Username or Password on Subscriber Binding

The following shows a subscriber log with a term_ec = 24 error, indicating an authentication error: the subscriber has the wrong user name or password on the subscriber binding:

[local]Redback#show subscribers log
32 IN Tue Mar 4 15:31:59.401541
IPC_ENDPOINT = DOT1Qd, MSG_TYPE = SESSION_DOWN, term_ec = 24
CCT_HANDLE = 3/2 vlan-id 105 
Internal Circuit = 3/2:103:63/1/2/48
aaa-idx = 300000e, extern-handle = 0, pvd_idx=4008002,
Event Code =0

2.2.3.2   Incorrect Context or Domain Name

[local]Redback#show subscribers log

The following example shows a subscriber log with a term_ec = 26 error code, indicating that the subscriber has an incorrect context or domain name or the context or the domain does not exist:

209 IN Mon Jan 2 17:52:17.837
IPC_ENDPOINT = PPPd, MSG_TYPE = SESSION_DOWN, term_ec = 26
Username = user@WRONG,
CCT_HANDLE = 13/1:1 vpi-vci 0 100
Internal Circuit = 13/1:1:63/1/2/11
aaa_idx = 1000005f, extern_handle = 0, pvd_idx = 40080002,
Event code = 0

2.2.3.3   Display Information About a Specific Subscriber

Use the show subscribers log username or show subscribers log session commands to see only the logs relevant to the problem session. Enter the subscriber argument as a structured subscriber username in the form subscriber@context. The following example shows how to display log information about subscriber user2 in the NiceService context using the grep options to search for a subscriber endpoint that is case insensitive:

[local]Redback#show subscribers log username user2@NiceService 
| grep options '-E -i' 'ipc_endpoint|--|\.'
------------------------------------------------------
0 IN Wed Aug 17 03:02:31.35980
IPC_ENDPOINT = PPPd, MSG_TYPE = AUTHEN_REQUEST,
----------------------------------------------------
1 OUT Wed Aug 17 03:02:31.40586 IPC_ENDPOINT = PPPd, 
MSG_TYPE= DB_RESPONSE,
----------------------------------------------------
2 IN Wed Aug 17 03:02:31.51376
IPC_ENDPOINT = PPPd, MSG_TYPE = SESSION_UP,
-----------------------------------------------------
3 OUT Wed Aug 17 03:02:31.51680
IPC_ENDPOINT = ISM-IF, MSG_TYPE = IF-BIND, 
-----------------------------------------------------
4 OUT Wed Aug 17 03:02:31.51714
IPC_ENDPOINT = ISM-CCT, MSG_TYPE = CCT-GEN-CFG,
-----------------------------------------------------

2.3   Step 3: Checking System Health

Use the show system alarm, command to check system health. For information about the show subscribers summary all command, see Displaying Summary Information About all Subscribers. For information about the show system alarm command, see the General Troubleshooting Guide.

2.3.1   Checking System Alarms

The show system alarm command displays system, card, port, channel or subchannel-level alarms. When you use the all option, the system displays alarms at all levels. For more information about this command, see the Command List. For more information about alarms and how to interpret them, see the SmartEdge router hardware guides. To enable the system alarm, enter the system alarm command (in global configuration mode). The default state of this alarm is disabled. This command enables the alarm for the air filter in the SmartEdge chassis, the redundancy alarm for SmartEdge systems with two controllers, and the transceiver alarm.

2.3.2   Displaying System Alarm

The following example shows an active alarm using the show system alarms command.

[local]Redback#show system alarms
Timestamp       Type    Source Severity Description
--------------------------------------------------------------------
Jan 19 10:37:58 chassis        Minor    Chassis power failure-side B

2.3.3   Displaying All System Alarms

The following example show active alarms on the fantray and chassis using the show system alarm command with the keyword all, which displays alarms at all levels:

[local]Redback#show system alarms all
Timestamp        Type    Source  Severity Description
----------------------------------------------------------------------------
Jun 12 17:42:56  chassis         Minor    Fan tray failure detected 
Jun 12 17:42:56  chassis         Minor    Fantray power-on diagnostic failed
Jun 12 17:42:56  chassis         Minor    Chassis power failure - side A1
Jun 12 17:42:56  chassis         Minor    Chassis power failure - side A2

2.3.4   Displaying Summary Information About all Subscribers

The following example shows how to display information about all subscribers in all contexts. The administrators must have system-wide privileges.

[local]Redback#show subscribers summary all
----------------------------------------------------------
Total=6 
Type       Authenticating       Active       Disconnecting
PPP                     0            0                   0
PPPoE                   0            1                   0
DOT1Q                   0            0                   0
CLIPs                   0            5                   0
ATM-B1483               0            0                   0
ATM-R1483               0            0                   0

2.4   Step 4: Checking for Configuration Errors

Use the show configuration context command and check for configuration errors listed in Table 5. Use this table as guide to troubleshoot common misconfiguration issues.

The following example shows how to display the NiceService context configuration:

[local]Redback#show configuration context NiceService

2.5   Step 5: Checking for Available IP Addresses

Use the show ip pool command to check the status of available IP addresses in the specified IP pool, in all IP pools in the specified interface, or in all IP pools in the current context or range.

The following example displays the status for all IP address pools in the ip-dial context, including a range of IP addresses for the isp1.net interface:

[local]Redback#context ip-dial
[ip-dial]Redback#show ip pool 

Interface "subscribers-am": 
 192.168.1.48  255.255.255.248  0 in use,  5 free, 3 reserved. 
Interface "subscribers-mr": 
 10.142.119.80 255.255.255.240  0 in use, 13 free, 3 reserved. 
Interface "subscribers-sz":
 192.168.2.0   255.255.255.0    0 in use, 253 free, 3 reserved.

Recommended Action: If you have a problem with the IP pool, check the IP pool configuration to see if there are enough addresses available for the subscribers. You might need to increase the pool range. The default subnet mask for the IP pool is /16, which supports a maximum of 65,533 subscribers.

2.6   Step 6: Checking Interface Connectivity

Use the ping, show port counters, and show circuit counters commands to check for interface connectivity.

2.6.1   Pinging the Interface

The following example shows you how to ping an interface.

[ISP1]Redback# ping 100.1.1.3
PING 100.1.1.3 (100.1.1.3): source 100.1.1.1, 36 data bytes,
timeout is 1 second  
----100.1.1.3 PING Statistics---- 5 packets transmitted,
5 packets received,0.0% packet loss round-trip min/avg/max/stddev = 
1.814/2.030/2.546/0.315 ms 
[local]Redback# ping atm channel end-to-end 13/1 vpi 1 vci 100 count 5
Sending 5, End-to-End F5 (Channel) cells on 13/1 :1 vpi 1 vci 100
Timeout is 2 seconds, Interval between Cells is 100 milliseconds
Success rate is 100.0 percent (5/5)

In the following example, look for packets being received that correspond to requests from the subscriber. If you do not use the detail or the live keywords, the counters are cached and are updated every 60 seconds.

[ISP2]Redback# show circuit counters 3/1 detail
please wait...
Circuit: 3/1 pppoe 1, Internal id: 1/1/4, Encap: 
ethernet-pppoe-ppp-combined
Packets                          Bytes
--------------------------------------------------------
Receive         :      43      Receive         :    4008
Receive/Second  :      0.05    Receive/Second  :    5.10
Transmit        :      44      Transmit        :    3890
Transmit/Second :      0.05    Transmit/Second :    5.10
IP Multicast Rcv:      0       IP Multicast Rcv:    0
IP Multicast Tx :      0       IP Multicast Tx :    0
Unknown Encaps  :      0       Unknown Encaps  :    0
Down Drops      :      0       Down Drops      :    0
Unreach Drops   :      0       Unreach Drops   :    0
Adj Drops       :      0       Adj Drops       :    0
WRED Drops Total:      0       WRED Drops Total:    0
Tail Drops Total:      0       Tail Drops Total:    0
PPP Counters cntrl:    3       cntrl           :    133
cntrl drops     :      0
retries         :      0
termreqs        :      0
PPPoE Counters
cntrl           :      2   
cntrl           :      120
session drops   :      0
PADT sent       :      0
PADR drops      :      0
PADI drops      :      0
PADT drops      :      0
bad code        :      0
Rate Refresh Interval :  60 seconds

2.6.2   Verifying Interfaces

Use the show ip interface brief to verify that the interfaces are up. This command displays the name, IP address, and other information (in brief) for all configured interfaces in the current context. Check the interface state to see if it is operational and binding to determine which physical circuit this interface uses to forward traffic. The binding has to be defined between a physical circuit (as port and PVC) and a logic interface in a context. For information about what to check on a port, see Checking Port Performance.

An interface can be in any of the following states:

• Unbound—The interface is not currently bound to any port or circuit. If the interface is unbound, for example, interface 12/1, it is a configuration error. The binding has to be configured between some physical circuit (like port or VLAN and interface 12/1. An interface can be unbound also if a subscriber with dynamic binding is not up, in which case you would troubleshoot the corresponding circuit.
• Bound—The interface is bound to at least one port or circuit; however, none of the bound circuits are up; therefore, the interface is not up. A bound state indicates that interface is correctly configured and that a problem exists on the L1/L2 level.
• Up—At least one of the bound circuits is in the up state; therefore, the interface is also up and traffic can be sent over the interface

The following example displays output from the show ip interface command with the brief keyword:

[local]Redback#show ip interface brief

Mon Jun 27 06:38:05 2005
Name      Address           MTU    State     Bindings
fe13/3    3.2.13.3/16       1500   Up        ethernet 13/3
fe13/4    4.2.13.4/16       1500   Up        ethernet 13/4
5/1       10.13.49.166/24   1500   Up        ethernet 5/1
12/1      10.1.1.1/16       0      UnBound
un1       (Un-numbered)     0      UnBound
lo1       100.1.1.1/16      1500   Up        (Loopback)

2.6.3   Checking Port Performance

Before you check the status of a port, you first need to understand the differences between “Admin state” and the “Line state”:

• Admin state—Refers whether the port has been brought up (using the no shutdown command) or is down (using the shutdown command). If the Admin state is shut down, the port is down.

  Recommended Action: Use the no shutdown command on the port to bring up the port.

• Line state—Refers to the physical state of the port.

  Recommended Action: When the Line state is down, use the following checklist:

If the Admin state is down, the Line state is always down. For the port to be up, the Admin state and Line state must both be up. To check the status of a port, issue the show port detail command. You must use the detail or live keyword to receive results in real time. For detailed information about each field displayed see the Command List.

Use the following table to determine whether a port is up or down.

In the following example, the status of the Ethernet port is down. Although the Ethernet port is in a no shutdown state and the Admin state is up, the cable has been unplugged from the Ethernet port 2/9 and, as a result, the Line state (the physical state) is down:

[local]redback#show port 2/9 detail

ethernet 2/9 state is Down 
Description                :
Line state                 : Down 
Admin state                : Up
Link Dampening             : disabled
Undampened line state      : Down
Dampening Count            : 0
Encapsulation              : ethernet
MTU size                   : 1500 Bytes
NAS Port Type              : 
MAC address                : 00:30:88:11:4d:37
Media type                 : 100Base-TX
Speed                      : 10 Mbps
Duplex mode                : half
Loopback                   : off
Active Alarms              : Link down

Use the show port counters live command to check port performance. By default, this command displays only summary counter information for all ports with their last known values, which have been cached; cached values are updated every 60 seconds. Use the live keyword to force the system to read and display live data for all summary counters except rate counters. If the counters are not increasing, packets are probably being dropped; use the show port counters detail command for detailed output. For detailed information about each field displayed, see the Command List.

Note:  

Depending on your configuration, it may take a few minutes to display information in real time when you use the live keyword.

For an ATM port:

• Verify that the ATM port is up using the show port slot/port detail command. When the port has a MAC address or linkgroup MAC address, verify that it is correct. For more information about this command, see the Command List.
• Verify the ATM PVC is up using the show atm pvc summary command. For "ppp llc", "ppp nlpid" or "ppp serial" encapsulations, confirm that the PPP daemon has the knowledge of this circuit. If there is a PVC down, use the show atm pvc all command to see a specific PVC. For more information about this command, see the Command List.
• Verify that ATM PVC is receiving packets using the show atm counter slot/port vpi vpi vci vci. If the PVC is not receiving packets, check the client settings and connection with the server.

[local]Redback#show port counters live

please wait...
Port            Type
5/3             ethernet
packets sent       : 0         bytes sent      : 0
packets recvd      : 0         bytes recvd     : 0
send packet rate   : 0.00      send bit rate   : 0.00
recv packet rate   : 0.00      recv bit rate   : 0.00
rate refresh interval : 60 seconds
7/1            ethernet
packets sent       : 13609     bytes sent      : 1292265
packets recvd      : 32791     bytes recvd     : 2035443
14/1           ethernet
packets sent       : 0         bytes sent      : 0
packets recvd      : 0         bytes recvd     : 0
send packet rate   : 0.00      send bit rate   : 0.00
recv packet rate   : 0.00      recv bit rate   : 0.00
rate refresh interval : 60 seconds

2.6.4   Monitoring Traffic on a Port

You can verify that you are receiving packets on your ports by running the monitor port counters command, which checks the current status of ports or channels and provides continuous status updates. This command can adversely impact system performance. Press Ctrl+C to exit monitoring mode. For detailed information about each field displayed, see the Command List.

The following example shows that no packets have been received during the 600 second interval on Ethernet port 5/1, which indicates there is an issue external to the SmartEdge router:

[local]Redback#monitor port counters 5/1

This may adversely impact system performance
% enter ctrl-C to exit monitor mode, monitor duration(sec): 
600 (00:00:02)

Port   Type      Pkts/Bytes Sent    Pkts/Bytes Received
5/1    ethernet          3                      0

2.6.5   Checking Circuit Performance

Use the show circuit counters command to display general counters and counters specific to a circuit type. Check for dropped packets in the Adj Drops, Down Drops, and Unknown Encaps fields. Use the show circuit counters ? command to display the various levels that you can check. For detailed information about each field displayed for, see the Command List.

The following example displays detailed information about circuit counters for a VLAN circuit. The values in the Adj Drops, Down Drops, and Unknown Encaps fields, which are highlighted in bold, have a value of zero (0), which indicates that the circuit is not dropping packets and is functioning correctly:

[local]Redback#show circuit counters 3/3 vlan-id 102 detail
[local]Redback#Circuit: 3/3 vlan-id 102, Internal id: 1/2/22,
Encap:ether-dot1q
Packets                      Bytes
--------------------------------------------------------------
Receive        : 26599       Receive         :        2297014
Receive/Second : 0.10        Receive/Second  :           8.60
Transmit       : 26538       Transmit        :        2285512
Xmits/Queue                  Xmits/Queue
 0             : 26538         0             :        2285512
 1             : 0             1             :              0
 2             : 0             2             :              0
 3             : 0             3             :              0
 4             : 0             4             :              0
 5             : 0             5             :              0
 6             : 0             6             :              0
 7             : 0             7             :              0
 8             : 0             8             :              0
 Transmit/Second :      0.10  Transmit/Second :          8.60
 IP Multicast Rcv:         0  IP Multicast Rcv:             0
 IP Multicast Tx :         0  IP Multicast Tx :             0
 Unknown Encaps  :         0  Unknown Encaps  :             0
 Down Drops      :         0  Down Drops      :             0
 Unreach Drops   :         0  Unreach Drops   :             0
 Adj Drops       :         0  Adj Drops       :             0
 ...

Use the show circuit counters handle handle detail command to see detailed information about whether the packets are received and sent to the circuit (only in case of unicast messages received from the client).

If there are no packets entering the circuit or port, investigate the fault at the circuit and port level.

The following example shows detailed information about a circuit that has unknown encapsulations :

[local]Redback#show circuit counters handle 255/22:1:26/1/1/10 detail | include Unk

Unknown Encaps  :       44  Unknown Encaps  :       7882
Unknown Encaps  :        0  Unknown Encaps  :          0
Unknown Encaps  :        0  Unknown Encaps  :          0

The following example displays detailed information about circuit counters for a PPPoE PVC. The values in the Adj Drops, Down Drops, and Unknown Encaps fields, which are highlighted in bold, have a value of zero (0), which indicates that the circuit is not dropping packets and is functioning correctly:

[local]Redback#show circuit counters pppoe detail

please wait...
Circuit: 13/1:1 vpi-vci 0 100, Internal id: 1/2/6,
Encap: atm-ppp-auto


Packets                        Bytes
----------------------------------------------------------
Receive          :   2550   Receive             :   140022
Receive/Second   :   0.50   Receive/Second      :    27.00
Transmit         :      45   Transmit           :     5309
Xmits/Queue                    Xmits/Queue
 0               :     45     0                 :     5309
 1               :      0     1                 :        0
 2               :      0     2                 :        0
 3               :      0     3                 :        0
 4               :      0     4                 :        0
 5               :      0     5                 :        0
 6               :      0     6                 :        0
 7               :      0     7                 :        0
 8               :      0     8                 :        0
Transmit/Second  :   	 0.00  Transmit/Second    :     0.00
IP Multicast Rcv :      0     IP Multicast Rcv   :        0
IP Multicast Tx  :      0     IP Multicast Tx    :        0
Unknown Encaps   :      0     Unknown Encaps     :        0
Down Drops       :      0     Down Drops         :        0
Unreach Drops    :      0     Unreach Drops      :        0
Adj Drops               0     Adj Drops          :        0

2.7   Step 7: Checking Bindings

Use the show bindings command to display the configured bindings for one or more subscribers, ports, channels, or PVCs on the system. Look at the Summary information to see if the total number bindings is bound. If not, check to see if the bound field increments. (Some of the bindings might be in transitory period.) If the bindings do not increment, use the show debug circuit command to gather more information about the circuit.

The following example displays all bindings in the current context (local):

[local]Redback#show bindings
Circuit                     State Encaps      Bind Type  Bind Name
1/1                         Up    cisco-hdlc  interface  toTokyo@London
1/2                         Up    cisco-hdlc  interface  toLondon@Tokyo
1/3                         Up    cisco-hdlc  interface  toLA1@NYC1
1/4                         Up    cisco-hdlc  interface  toNYC1@LA1
2/1:5                       Up    cisco-hdlc  interface  toNYC2@London
2/1:6                       Up    cisco-hdlc  interface  toNYC1@London
2/1:7                       Up    cisco-hdlc  interface  toLA1@Tokyo
2/1:8                       Up    cisco-hdlc  interface  toLA2@Tokyo
2/1                         Up    cisco-hdlc                  
2/2:5                       Up    cisco-hdlc  interface  toLondon@NYC2
2/2:6                       Up    cisco-hdlc  interface  toLondon@NYC1
2/2:7                       Up    cisco-hdlc  interface  toTokyo@LA1
2/2:8                       Up    cisco-hdlc  interface  toTokyo@LA2
2/2:15                      Up    cisco-hdlc                  
2/2:16                      Up    cisco-hdlc                  
2/2                         Up    cisco-hdlc                  
5/1                         Down  ethernet                    
6/1:1 vpi-vci 1 101         Down  bridge1483  interface  internal@London
6/1:1 vpi-vci 1 102         Down  bridge1483  interface  internal@Tokyo
6/1:1 vpi-vci 4 4           Down  multi1483                   
6/1:1 vpi-vci 44 45         Down  multi1483                   
6/1:1 vpi-vci 55 66         Down  multi1483                   
7/1                         Up    ethernet     interface  adm@local
10/1:1                      Down  frame-relay                 
10/1:1 dlci 0               Down  frame-relay                 
10/1:1 dlci 1023            Down  frame-relay                 
10/1:1 dlci 16              Down  frame-relay                 
10/1:3                      Down  cisco-hdlc                  
11/1                        Down  cisco-hdlc                  
12/1                        Down  ethernet     interface  toNYC1@NYC2
12/1 vlan-id 1              Down  dot1q multi                 
12/2                        Down  ethernet     interface  toNYC2@NYC1
12/3                        Down  ethernet     interface  toLA2@LA1
12/4                        Down  ethernet     interface  toLA1@LA2
GRE 1.2.3.4 key 1           Down  gre                         
Link share ethernet         Down  ethernet
Summary:
   total: 38              up: 19            down: 19     
   bound: 19         unbound: 19         no-bind: 19     
    auth: 0        interface: 19      subscriber: 0      
     atm: 5            chdlc: 20           dot1q: 1        ether: 7      
      fr: 4              gre: 1             mpls: 0          ppp: 0      
   pppoe: 0            clips: 0

The following example displays binding information for all PVCs configured with the bind interface command for port 1 on the card in slot 2:

[local]Redback(config-ctx)#show bindings 2/1 interface
Circuit         State Encaps         Bind Type  Bind Name
2/1:5           Up    cisco-hdlc     interface  toNYC2@London
2/1:6           Up    cisco-hdlc     interface  toNYC1@London
2/1:7           Up    cisco-hdlc     interface  toLA1@Tokyo
2/1:8           Up    cisco-hdlc     interface  toLA2@Tokyo
Summary:
   total: 4             up: 4             down: 0      
   bound: 4        unbound: 0          no-bind: 0      
    auth: 0      interface: 4       subscriber: 0      
     atm: 0          chdlc: 4            dot1q: 0     ether: 0      
      fr: 0            gre: 0             mpls: 0     ppp: 0      
   pppoe: 0          clips: 0

The following example displays all bindings for all Frame Relay PVCs for port 1 on the card in slot 10:

[local]Redback(config-ctx)#show bindings 10/1 fr

Circuit               State Encaps        Bind Type  Bind Name
10/1:1                Down  frame-relay                 
10/1:1 dlci 0         Down  frame-relay                 
10/1:1 dlci 1023      Down  frame-relay                 
10/1:1 dlci 16        Down  frame-relay                 
Summary:
   total: 4               up: 0             down: 4      
   bound: 0          unbound: 4          no-bind: 4      
    auth: 0        interface: 0       subscriber: 0      
     atm: 0            chdlc: 0            dot1q: 0   ether: 0      
      fr: 4              gre: 0             mpls: 0     ppp: 0      
   pppoe: 0            clips: 0

2.8   Step 8: Checking Licenses

Use the show licenses command to display a list of software licenses and their configuration status. To see if you are operating within the licensed limits (for example, number of subscribers), issue the show subscribers summary all command. Some licenses have no limits. If the feature is enabled, check that the correct license is installed.

The following example displays configured software licenses:

[local]Redback#show licenses
Software Feature                License Configured
--------------------------      ------------------
l2tp all                        YES
subscriber active 8000          YES
 Total active subscriber license configured 8000 

The following example displays all software licenses and their configuration status:

[local]Redback#show licenses all
Software Feature                License Configured
--------------------------      ------------------
subscriber dynamic-service      NO
l2tp all                        YES
mpls                            NO
subscriber high-availibility    NO
subscriber active 8000          YES
subscriber bandwidth            NO
Total active subscriber license configured 8000 

2.9   Step 9: Checking Authentication

The SmartEdge router uses RADIUS servers to authenticate subscribers. The SmartEdge RADIUS client passes subscriber information to designated RADIUS servers, and then acts on the returned response. RADIUS servers receive user connection requests, authenticate the user, and then return all configuration information required for the client to deliver service to the subscriber.

A number of counters are incremented whenever a RADIUS server encounters errors. For example, if a RADIUS server rejected authentication requests because it is too busy, you can check the authen fail due to throttling counter. The output from the show subscriber log command is also useful for checking authentication requests. We recommend that you use the show subscribers log username subscriber or show subscribers log session commands to view only the logs relevant to the problem during the session. For an example of the show subscribers log username command, see Displaying Subscriber Logs.

Use the following commands to troubleshoot authentication problems, such as an incorrect username, password, or an unstructured username:

• show subscriber log
• show circuit slot/port vpi-vci
• debug aaa exception

2.9.1   Displaying AAA Logs

Use the show subscribers log command to display the AAA log. The output tracks inbound and outbound messages from the AAAd process.

The following example displays an unknown circuit from the AAA log:

[local]Redback#show subscribers log
---------------------------------------------------------
Total log size : 25000
Next log index : 1893
Log wrapped    : 58 time(s)
---------------------------------------------------------
0       OUT     Thu Sep 11 17:33:36.548471
IPC_ENDPOINT = ISM-IF, MSG_TYPE = IF-UNBIND,
Username = user2@NiceService,
CCT_HANDLE = Unknown circuit
Internal Circuit = 2/14:1023:63/6/2/27408
aaa_idx = 10056cc9, extern_handle = 4f, pvd_idx = 4008000d,
Event code = 0
---------------------------------------------------------
1       IN      Thu Sep 11 17:33:36.548486
IPC_ENDPOINT = PPPd, MSG_TYPE = SESSION_DOWN, term_ec = 142
terminate cause = No traffic within idle timeout period
Username = user2@NiceService,
CCT_HANDLE = Unknown circuit
Internal Circuit = 2/14:1023:63/6/2/27409
aaa_idx = 10056cca, extern_handle = 50, pvd_idx = 4008000d,
Event code = 0

Recommended Action: Use the show circuit command to obtain more information about the unknown circuit.

2.9.2   Debugging AAA

Use the debug aaa exception command to display a AAA packet or function that unexpectedly ends a task; for example, an invalid password or username during authentication. In the output, all debug messages for successful authentication are filtered out, and only the error-condition debug logs are displayed—particularly useful when many subscribers are authenticating simultaneously.

Use the following AAA troubleshooting check list to check for common AAA configuration issues.

The following example shows how to display the AAA log, which shows a subscriber with incorrect credentials:

[local]Redback#debug aaa exception
Feb 6 15:47:15: [13/1:1:63/1/2/11]: 
%AAA-7-EXCEPT1: aaa_idx 10000029:
Cannot bind subscriber user2@NiceService to valid context
Feb 6 15:47:15: [13/1:1:63/1/2/11]: %AAA-7-EXCEPT1:
aaa_idx 10000029:
aaa_remove_session_from_trees: remove session that 
is not bound to any context yet

Recommended Action: Make sure that the subscriber is correctly configured. To determine the cause of the exception, use the debug aaa all command.

2.9.3   Displaying Information About a VPI and VCI for an ATM PVC

Use the show circuit slot/port vpi-vci to display information about VPI and VCI for an ATM PVC. In the following example, the circuit is down because it is unbound (no-bind: 1):

[local]Redback#show circuit 4/2:1 vpi-vci 200 20
Circuit               Internal Id  Encap     State Bound to
4/2:1 vpi-vci 200 20  1/2/27       atm-cell  Down
Summary:
  total: 1
     up: 0             down: 1
  bound: 0          unbound: 1
   auth: 0        interface: 0      subscriber: 0  bypass: 0
   no-bind: 1           atm: 1           chdlc: 0  dot1q: 0
     ether: 0            fr: 0             gre: 0
      mpls: 0           ppp: 0           pppoe: 0
     clips: 0          vpls: 0            ipip: 0
     ipsec:      ipv6v4-man: 0     ipv6v4-auto: 0

Recommended Action:

1. Check if the port is down.
2. Check for a configuration mismatch.
3. If the configuration is correct, check to see why the circuit is down. Check for authentication issues using the following commands: debug aaa authen, and debug aaa auth and debug aaa exception.
4. If authentication is through RADIUS, verify that the RADIUS server is functioning using the show radius server and show radius statistics commands. For information about these commands, see Troubleshooting RADIUS.
5. Use the debug pppoe exception command to see if there are any unexpected events with PPPoE.

3   Troubleshooting PPP

This section describes how to troubleshoot PPP. For information about troubleshooting PPPoE, see Troubleshooting PPPoE. For information about operational commands, see the Command List.

The following is a sample PPP configuration for the SmartEdge router in context isp1:

context isp1
 interface forPPP-clients multibind
  ip address 1.1.1.1/24
  ip pool 1.1.1.0/24
 interface toInternet 
  ip address 2.1.1.1/24
 ip route 0.0.0.0/0 2.1.1.254
 aaa authentication subscriber local
 subscriber default
  ip address pool
 subscriber name pppoa1
  password test1
 subscriber name pppoe1
  password test2
 subscriber name vlan1
  password test3
 atm profile ubr
  shaping ubr
!
port atm 1/1
 no shutdown
 atm pvc 1 32 profile ubr encapsulation ppp
  bind authentication chap pap
 atm pvc 1 33 profile ubr encapsulation pppoe
  bind authentication chap pap
!
port ethernet 2/1
 encapsulation dot1q
 dot1q pvc 32 encapsulation pppoe
  bind authentication chap pap
!
port ethernet 2/4
 no shutdown
 bind interface toInternet isp1.net

3.1   PPP Troubleshooting Tasks

Use Table 9 as a guide to troubleshooting PPP issues. Before you begin, get a description of the problem and check if you made any recent changes or upgrades to the network.

Note:  

When working with technical support representatives, run the show tech-support command on your router and have it available to assist in troubleshooting.

3.2   Step 1: Navigating to the Correct Context

Use the show context all command and display all the contexts on your router and then navigate to the context you want to troubleshoot—in this case, the isp1 context.

The following example shows how to view all contexts on your router and then navigate to context isp1:

[local]Redback#show context all
Context Name    Context ID    VPN-RD   Description
--------------------------------------------------
local            0x40080001
isp1             0x40080002

[local]Redback#
[local]Redback#context isp1
[isp1]Redback#

3.3   Step 2: Checking Port Counters

Use the show port counter live command to check port performance. For information about this command, see Checking Port Performance.

For an ATM port:

• Verify that the ATM port is up using the show port slot/port detail command. When the port has a MAC address or linkgroup MAC address, verify that it is correct.
• Verify the ATM PVC is up using the show atm pvc summary command. For "ppp llc", "ppp nlpid", or "ppp serial" encapsulations, confirm that the PPP daemon recognizes this circuit. If a PVC is down, use the show atm pvc all command to see all PVCs.
• Verify that ATM PVC is receiving packets by using the show atm counter slot/port vpi vpi vci vci command. If the PVC is not receiving packets, check the client settings and connection with the server.

3.4   Step 3: Verifying that PPP is Receiving Packets

Use the show ppp counters to check that the PPP daemon is receiving packets. This command displays statistics for PPP packets and session counters on the system.

The following example shows how to display the context-specific PPP counters for the current context:

[local]Redback#show ppp counters context
Last cleared: Never
received: bytes 260, packets 26,
unsupported packets 0
sent: bytes 260, packets 26
LCP echo request : received 26, sent 0, dropped 0
LCP echo response : received 0, sent 26, dropped 0
LCP protocol reject : received 0, sent 0, dropped 0

The clear option provides more recent PPP counter information. Use the clear option after the show ppp counters or show ppp counters detail commands to clear the counters and recollect them after 30 seconds. After 30 seconds, look at what caused PPP to go down. If you want to keep a record of your show counters for an extended period of time, capture your log information and save it before you use this option. Otherwise, use the show ppp counters command.

The following example shows how to display global PPP counters:

[isp1]Redback#show ppp counters clear 
Current time: Mon Apr 23 07:31:45 2007
 Last cleared: Never Packet
Wed Jun 30 21:56:07 2005
Packet--------------------------------------------------------
In                          285     Out                    287
Session-------------------------------------------------------
LCP Up                       72     LCP Down                68
IPCP Up                      12     IPCP Down                6
Authen Success                0     Authen Failure           0
Session Up                    0     Session Down             0
SessionControl------------------------------------------------
Starting                      0     Authenticating           0
Pended (current)              0     Pended (total)           0
Packet Drop
Session pended                0     At Limit                 0
Timeout-------------------------------------------------------
ConfReq                      85     TermReq                 19
CHAP Challenge                0     UPAP Listen              0
PacketDropIn--------------------------------------------------
Session is Down              17     Bad FSM State           32

The following example shows how to display detailed information for global PPP counters:

[isp1]Redback#show ppp counters detail clear
Packet--------------------------------------------------------
In                           40 Out                         40
ConfReq                      24  ConfReq                    10
ConfAck                      10  ConfAck                    10
ConfNak                       0  ConfNak                     4
ConfRej                       0  ConfRej                    10
TermReq                       4  TermReq                     2
TermAck                       2  TermAck                     4
Authen Proto                  6  Authen Proto                6
other                         0  other                       0
Session-------------------------------------------------------
LCP Up                        6 LCP Down                     6
 IPCP Up                      4 IPCP Down                    4
 Authen Success               4 Authen Failure               2
 Session Up                   4 Session Down                 6
SessionControl------------------------------------------------
 Starting                     0 Authenticating               0
 Pended (current)             0 Pended (total)               0
 Packet Drop
 Session pended               0 At Limit                     0
Timeout-------------------------------------------------------
 ConfReq                      0 TermReq                      4
 CHAP Challenge               0 UPAP Listen                  0
PacketDropIn--------------------------------------------------
 Session is Down              1 Bad FSM State                0
DownCause-----------------------------------------------------
 Rcvd TermReq                 4 Rcvd PPPoE PADT              0
 No ConfReq Resp              0 No Echo Resp                 0
 Authen Failed                2 Session Down                 0
 LCP Down                     0 Circuit Down                 0
 Port Down                    0 Port Delete                  0

3.5   Step 4: Checking PPP Circuit Counters

Use the show circuit counters ppp detail command to check the status of the PPP protocol.

The following example shows you how to check the PPP state. The PPP Cntrl counters should increase. The remaining counters should be zero (0).

[ips1]Redback# show circuit counters ppp detail
Circuit: 1/4 vlan-id 10 pppoe 62, Internal id: 6/2/62, Encap:
ether-dot1q-pppoe-ppp
Packets                                 Bytes     
-------------------------------------------------------------------
Receive         :                   50  Receive         :      3950
Receive/Second  :                 0.00  Receive/Second  :      0.00
Transmit        :                   51  Transmit        :      3968
Xmits/Queue                             Xmits/Queue 
0             :                   51    0             :       3968
1             :                    0    1             :          0
2             :                    0    2             :          0
3             :                    0    3             :          0
4             :                    0    4             :          0
5             :                    0    5             :          0
6             :                    0    6             :          0
7             :                    0    7             :          0
Xmit Q Deleted  :                  0  Xmit Q Deleted  :          0
Transmit/Second :                 0.00  Transmit/Second :       0.00
IP Multicast Rcv:                    0  IP Multicast Rcv:        0
IP Multicast Tx :                    0  IP Multicast Tx :        0
Unknown Encaps  :                    0  Unknown Encaps  :        0
Down Drops      :                    0  Down Drops      :        0
Unreach Drops   :                    0  Unreach Drops   :        0
Adj Drops       :                    0  Adj Drops       :        0
WRED Drops Total:                    0  WRED Drops Total:        0
WRED Drops/Queue                        WRED Drops/Queue   
0             :                    0    0             :          0
1             :                    0    1             :          0
2             :                    0    2             :          0
3             :                    0    3             :          0
4             :                    0    4             :          0
5             :                    0    5             :          0
6             :                    0    6             :          0
7             :                    0    7             :          0
Tail Drops Total:                    0  Tail Drops Total: 0
Tail Drops/Queue                        Tail Drops/Queue 
0             :                    0    0             :          0
1             :                    0    1             :          0
2             :                    0    2             :          0
3             :                    0    3             :          0
4             :                    0    4             :          0
5             :                    0    5             :          0
6             :                    0    6             :          0
7             :                    0    7             :          0
PPP Counters
Cntrl Rcv       :                    4  Cntrl Rcv       : 190
Cntrl Tx        :                    4  Cntrl Tx        : 170
Cntrl Drops Rcv :                    0
Retries Rcv     :                    0
Termreqs Rcv    :                    0
PPPoE Counters
Cntrl           :                    0  Cntrl           : 0
Session Drops   :                    0
PADT Sent       :                    0
PADR Drops      :                    0
PADI Drops      :                    0
PADT Drops      :                    0
Bad Code        :                    0
Rate Refresh Interval : 60 seconds

3.6   Step 5: Checking the PPP State on the Circuit

Use the show circuit detail or show circuit handle handle detail commands to check the PPP state on the circuits. Make sure the encapsulation type is correct.

Check the following:

• Verify the circuit is up and the PPP LCP and the PPP ICP states are Opened
• Verify the encapsulation type. It should be *-pppoe-ppp-combined
• Verify the MTU size. It should be 1492.

[isp1]Redback# show circuit detail
Circuit: 3/1 pppoe 1, internal id: 1/1/7015, state: Up
interface bound : pool@test
subscriber bound : pppoa1@isp.net
bind type : chap pap
admin state : 1 hardware address : 00:30:88:00:77:0c
media type : ethernet encap type : ethernet-pppoe-ppp-combined
mode type : 0x2 port type : etherne
mtu size : 1492 cfg mtu size : 1500
ipv6 mtu size : 1500 ipv6 cfg mtu size : 1500
cct speed : 100000 cct rx speed : 0
cct flags (attr) : 0x1
slot mask : 0x0 ppa cct clear : FALSE
if flags : 0x0
profile id : 0 version : 207874
PPP OSINLCP State : Initial PPP MPLSCP State : Initial
PPPOE State : READY
internal handle : 3/1:1023:63/1/1/7015

Note:  

To find a circuit handle, use the show circuit circuit-type detail | grep [ip-address | username | mac-address] command. When you use this command, make sure the encapsulation type is correct.

3.7   Step 6: Displaying PPP Summary Information

Use the show ppp summary command to display summary output for PPP sessions in the current context. When you use this command, for "ppp llc", "ppp nlpid", or "ppp serial" encapsulations, make sure that the PPP daemon recognizes this circuit . You can also use the show ppp down command to specify that only PPP sessions for which the LCP is in the INITIAL, CLOSED, or STOPPED state are displayed:

[isp1]Redback#show ppp summmary
Wed Jun 30 21:54:29 2005
Number    LCP    IPCP   NLCP  MPLSCP
Circuit Type   Circuit   Open   Open   Open  Open
------------   -------   ----   ----   ----  ------
mp circuits   2         2      2      0     0
ppp circuits  2         2      2      0     0
Total circuits: 4  up: 4  down: 0

If the circuits are down, use the show circuit ppp down detail command to collect more information about the issue.

[isp1]Redback#show circuit ppp down detail
Circuit: 4/1:1 vpi-vci 1 100, internal id: 1/2/8388608, state: Down
------------------------------------------------------------------------------
interface bound   :
subscriber bound  :
bind type         : chap pap
admin state       : 1            hardware address  :
00:30:88:00:37:11
media type        : atm          encap type        : atm-ppp-vcmux
mode type         : 0x2          port type         : atm
mtu size          : 4470         cfg mtu size      : 4470
ipv6 mtu size     : 4470         ipv6 cfg mtu size : 4470
cct speed         : 149760       cct rx speed      : 149760
cct flags (attr)  : 0x1
slot mask         : 0x0          parent slot mask  : 0x0 ppa cct clear: FALSE
if flags          : 0x0          aaa index         : 0x0
profile id        : 1342193665   version           : 987
h node id         : 0
lg_id             : 0            spg_id            : 0
PPP LCP State     : Initial      PPP IPCP State    : Initial
PPP OSINLCP State : Initial      PPP MPLSCP State  : Initial
internal handle   : 4/1:1:63/1/2/8388608

3.8   Step 7: Checking Interfaces

Use the show ip interface brief to verify that the interfaces are up. This command displays the name, IP address, and other information (in brief) for all configured interfaces in the current context.

The following example displays output from the show ip interface command with the brief keyword:

[isp1]Redback#show ip interface brief
Mon Jun 27 06:38:05 2005
Name       Address          MTU    State      Bindings
fe13/3     3.2.13.3/16      1500   Up         ethernet 13/3
fe13/4     4.2.13.4/16      1500   Up         ethernet 13/4
5/1        10.13.49.166/24  1500   Up         ethernet 5/1
12/1       10.1.1.1/16      0      UnBound
un1        (Un-numbered)    0      UnBound
lo1        100.1.1.1/16     1500   Up         (Loopback)

Recommended Action: If the interfaces are down, see Verifying Interfaces and Checking Port Performance.

3.9   Step 8: Checking PPP Bindings

Use the show bindings command to verify your bindings. For information about this command, see Section 2.7.

3.10   Step 9: Checking Subscribers

Use the show subscribers commands to check your subscribers. Following are some examples of how to use show subscribers commands:

For more information about how to use the show subscribers command, Displaying Information About My Subscribers.

3.11   Step 10: Displaying Subscriber Routes

Use the show ip route subscriber command to check where the subscriber is terminated. Make sure the IP address negotiated during the IPCP stage is correctly installed in the routing table.

[ips1]Redback# show ip route subscriber
Codes: C - connected, S - static, S dv - dvsr, R - RIP, e B - EBGP, i B
 - IBGP
    A,H - derived hidden
    O   - OSPF, IA - OSPF inter area, N1  - OSPF NSSA external type 1
    N2  - OSPF NSSA external type 2,  E1  - OSPF external type 1
    E2  - OSPF external type 2
    I   - IS-IS, L1 - IS-IS level-1,  L2  - IS-IS level-2
    IPH - IP Host, SUB A - Subscriber address, SUB S - Subscriber
 static
       A - Derived Default
       >   - Active Route
Type    Network       Next Hop   Dist  Metric  UpTime       Interface
> SUB A 20.1.1.1/32                15      0   00:01:08     LNS1
> SUB A 20.1.1.2/32                15      0   00:01:08     LNS1
> SUB A 30.1.1.0/32                15      0   00:01:08     LNS1

3.12   Step 11: Checking PPP Process

Use the show process ppp command to verify that the PPP process is running. For detailed information, use the detail keyword. If the PPP process is not running, use the show crashfiles command to check if there is a core dump. If there are crash files, contact technical support.

[isp1]Redback#show process pppNAME PID  SPAWN MEMORY TIME         %CPU  STATE UP/DOWN
ppp  166  3     3436K  00:00:00.59  0.00% run   00:02:20

[local]Redback#show crashfiles
4844 Jul 4 16:02 /md/pppd_43.mini.core
5944456 Jul 4 16:02 /md/pppd_43.core
4812 Jul 4 16:03 /md/pppd_526.mini.core
5923992 Jul 4 16:03 /md/pppd_526.core

3.13   Step 12: Checking the RADIUS Server

If you have configured a RADIUS server, make sure the RADIUS attributes are configured correctly for your clients. For information about checking the RADIUS Server, see Section 16, Troubleshooting the RADIUS Server.

3.14   Step 13: Checking the PPP Configuration

Use the show configuration ppp command and check for configuration issues listed in Table 11. Use this table as guide to troubleshoot common misconfiguration issues.

Note:  

Verify that the PPA circuit is receiving packets by using the show circuit counter slot/port vpi vpi vci vci detail command. For an ATM card, if the packets are not received, investigate the packet and cell drop between Segmentation and Reassembly (SAR) and the PPA.

3.15   Step 14: Debugging PPP

Use the debug ppp command to enable the generation of debug messages for various types of PPP events on the system.

(1)  The SmartEdge 100 router does not support the standby keyword.

(2)  The SmartEdge100 router does not support the switchover keyword.

To store debug messages in the system log buffer, use the logging debug command (in global configuration mode). Use the show log command (in exec mode) to display these stored debug messages. For information about the show log command, see the Command List.

To display messages in real time, use the logging console command (in context configuration mode) if you are connected to the system through the console port. Or, use the terminal monitor command (in exec mode) if you are connected to the system through a Telnet or Secure Shell (SSH) session.

Use the no form of this command to disable the generation of debug messages for PPP events.

3.15.1   Debugging PPP Packets

Use the debug ppp packet command to determine why a PPP session is not coming up. It shows all the negotiation while keeping debugging to a minimum. Check for any increasing exception counters, such as At Limit, No Circuit, and No Unit. These may indicate that the PPP daemon does not recognize the circuit where packets were received, or another software problem.

Monitoring the following:

• LCP messages
  • in <<, or out >> LCP packets for information
  • LCP negotiation methods: PAP, CHAP or none
  • LCP MRU size
  • ConfReq and ConfAck ID matching messages
  • PPP-ERR

  Recommended Action: Enable the debug ppp all command to obtain detailed information about the issue.

• Authentication messages
  • PAP or CHAP AuthReq and AuthAck
  • Authentication ID number matching
  • Username and password.

    Recommended Action: If any of these messages fail, debug AAA. For information about troubleshooting RADIUS, see Troubleshooting RADIUS.

• IPCP negotiation messages
  • IPCP negotiation options (IP Address)
  • IPCP ConfReq and ConfAck
  • Subscriber IP Address is within subnet

The following example shows you how to use the debug ppp packet command.

[isp1]Redback# debug ppp packet
[isp1]]Redback#terminal monitor
[isp1]Redback##show debug // Display enabled debug commands.

3.15.2   PPP Circuit-Level Debug Commands

The following example show how to enable the generation of all PPP debug messages for all circuits on port 1 on the traffic card in slot 14:

[isp1]Redback#debug circuit handle 14/1:1023:63/2/2/1
[local]Redback#debug circuit ppp packet

The following example show how to enable the generation of all PPP debug messages circuit handle 14/1:1023:63/2/2/1:

[isp1]Redback#debug circuit handle 14/1:1023:63/2/2/1
[local]Redback#debug circuit ppp all
[isp1]Redback#show debug circuit //Shows circuits that are being debugged.  
Circuit debugging:                 
  14/1:1023:63/2/2/1

Note:  

To find a circuit handle, use the show circuit circuit-type detail | grep [ip-address | username | mac-address] command.

4   Troubleshooting PPPoE

This section describes how to troubleshoot PPPoE. For information about troubleshooting PPP, see Troubleshooting PPP. For information about operational commands, see the Command List.

4.1   Overview of PPPoE

There are four steps to PPPoE session setup based on RFC 2516, A Method for Transmitting PPP Over Ethernet (PPPoE):

1. The host broadcasts a PPPoE Active Discovery Initiation (PADI) packet.
2. When the access concentrator receives a PADI that it can serve, it replies by sending a PPPoE Active Discovery Offer (PADO) packet to the host.
3. Because the PADI was broadcast, the host may receive more than one PADO packet. The host chooses a PADO packet based on the AC name or the services offered. The host then sends a single PPPoE Active Discovery Request (PADR) packet to the access concentrator that it has chosen.
4. When the access concentrator receives a PADR packet, it prepares to begin a PPP session. It generates a unique session ID for the PPPoE session and replies to the host with a PPPoE Active Discovery Session-confirmation (PADS) packet.

When a host initiates a PPPoE session, it must first perform discovery to identify the Ethernet MAC address of the peer and establish a PPPoE session ID. Although PPP defines a peer-to-peer relationship, discovery is inherently a client/server relationship. In the discovery process, a host (the client) discovers an access concentrator (the server).

Depending on the network topology, there may be more than one access concentrator with which the host can communicate. The discovery stage allows the host to discover all access concentrators and then select one. When discovery is completed, both the host and the selected access concentrator have the information required to build a PPPoE connection.

4.2   Before You Begin

Before you begin, get a description of the problem and check if you made any recent changes or upgrades to your network, and then check whether the issue is at the PPPoE or PPP level. Then, use the following commands to determine if the issue is at the PPPoE or PPP level. If the issue is at the PPPoE level, troubleshoot PPPoE, if not, troubleshoot PPP.

1. show process ppp
2. show process pppoe
3. debug pppoe exception
4. debug ppp exception
5. show ppp counter clear
6. show ppp counter detail clear
7. show pppoe counter clear
8. show pppoe counter detail clear

For information about Troubleshooting PPP, see Troubleshooting PPP.

Note:  

When working with technical support representatives, use the show tech-support command on your router and have the output available for your local technical support representative to assist in troubleshooting.

4.3   PPPoE Sample Configuration

The following is a sample configuration on a SmartEdge 800 router in context isp. This configuration maps to the PPPoE troubleshooting tasks.

config
!
context isp
domain isp.net
!
interface pppox multibind
ip address 160.1.1.1/16
ip pool 160.1.0.0/16
!
subscriber pppoeoa
password test1
ip address pool
!
subscriber vlan
password test2
ip address pool
!
subscriber untag1
password test3
ip address pool
!
subscriber untag2
password test4
ip address pool
!
atm profile ubr
shaping ubr
!
port atm 1/1
no shutdown
atm pvc 1 32 profile ubr encapsulation pppoe
bind authentication pap
!
port ethernet 2/1
no shutdown
encapsulation dot1q
dot1q pvc 32
bind auth pap
!
port ethernet 2/2
no shutdown
encapsulation pppoe
bind auth pap max 10
!

4.4   PPPoE Troubleshooting Tasks

Use the following table as a guide to troubleshooting general PPPoE issues. Before you begin, get a description of the problem and check if you made any recent changes or upgrades to the network.

Note:  

When working with technical support representatives, issue the show tech-support command on your router and have the output available for your technical support representatives to assist in troubleshooting.

4.5   Step 1: Navigating to the Correct Context

Use the show context all command and display all the contexts on your router and then navigate to the context you want to troubleshoot, in this case, the isp context.

The following example shows how to view all contexts on your router and then navigate to context isp:

[local]Redback#show context all
Context Name Context ID VPN-RD Description
------------------------------------------
local 0x40080001
isp 0x40080002
[local]Redback#
[local]Redback#context isp
[isp]Redback#

4.6   Step 2: Checking Port Counters

Use the show port counters live command to check the port counters in real time. Use the detail keyword to display detailed port counter information. For more information about this command, see Checking Port Status.

[isp]Redback#show port counters live

please wait...
Port Type
1/1 ATM
packets sent : 15000 bytes sent : 20000
packets recvd : 10000 bytes recvd : 50000
send packet rate : 0.00 send bit rate : 0.00
recv packet rate : 0.00 recv bit rate : 0.00
rate refresh interval : 60 seconds
2/1 ethernet
packets sent : 25000 bytes sent : 40000
packets recvd : 60000 bytes recvd : 80000
send packet rate : 0.00 send bit rate : 0.00
recv packet rate : 0.00 recv bit rate : 0.00
rate refresh interval : 60 seconds
2/2 ethernet
packets sent : 29000 bytes sent : 20000
packets recvd : 70000 bytes recvd : 30000
send packet rate : 0.00 send bit rate : 0.00
recv packet rate : 0.00 recv bit rate : 0.00
rate refresh interval : 60 seconds

4.7   Step 3: Checking PPPoE Counters

Use the show pppoe counters command to display summary or detailed statistics for all PPPoE-encapsulated circuits. For example:

• Packet counts for all PPPoE events, including RX and TX PPPoE PAD counters
• PPPoE invalid discovery packet counters
• PPPoE virtual circuit counters
• PPPoE discovery processing counters
• PPPoE PADM error counters

Make sure that you are receiving traffic; check for dropped packets. The sent and receive packets should closely match.

The clear option provides more recent PPPoE counter information. Use the clear option after the show pppoe counters or show pppoe counters detail commands to clear the counters and recollect them after 30 seconds. After 30 seconds, look at what caused PPPoE to go down. If you want to keep a record of the show pppoe counters for an extended period of time, capture your log information and save it before you use this option. Otherwise, use the show pppoe counters command.

When you use the show pppoe counters command, make sure you use the following checklist:

If the PPPoE subscribers cannot connect, issue the show pppoe counters detail command and use the following checklist.

The following example shows how to display summary statistics:

[local]Redback#show pppoe counters clear 

Wed Jun 30 01:37:25 2005
PPPoE PAD counters:
-------------------------------------------------------
sent packets        : 4000         recv packets : 4000
dropped packets     : 0
PADI packets        : 2000         PADO packets : 2000
PADR packets        : 2000         PADS packets : 2000
PADT packets        : 0            PADM packets : 0
PADN packets        : 0

The following example shows how to display detailed statistics:

[local]Redback#show pppoe counters detail clear  

Wed Jun 30 08:30:40 2005
PPPoE PAD counters:
--------------------------------------------------------------
sent packets        : 4000      recv packets            : 4000
dropped packets     : 0         
PADI packets        : 2000      PADO packets            : 2000
PADR packets        : 2000      PADS packets            : 2000
PADT packets        : 0         PADM packets            : 0   
PADN packets        : 0         
PPPoE invalid discovery packet counters: 
--------------------------------------------------------------
invalid version/type     : 0   invalid length           : 0   
invalid tag length       : 0   unknown code             : 0   
PADIs non-zero sess-id   : 0   PADRs non-zero sess-id   : 0   
PADT, bad MAC addr       : 0   bad encaps               : 0   
PADR, max sess reached   : 0   PADR, same MAC           : 0   
tags not added, large pkt: 0   recv on down circuit     : 0   
invalid tag name         : 0   invalid tag name accepted: 0   
circuit not created      : 0   circuit not init         : 0   
packet on virtual circuit: 0   non subscriber circuit   : 0   
unknown circuit          : 0   proc restart drops       : 0   
PPPoE virtual circuit counters:
---------------------------------------------------------------
created virtual circuits : 0    deleted virtual circuits : 0   
combined circuits used   : 4000 combined circuits reset  : 2000
create failed            : 0    delete failed            : 0   
create fail, rcct used   : 0    create fail, no cct      : 0   
create fail, cct init    : 0    create fail, vcct exists : 0   
circuit lookup failures  : 0         
PPPoE PADM error counters:
---------------------------------------------------------------
malformed URLs           : 0    too long expanded URLs   : 0   
too long MOTMs           : 0    bad expansion char       : 0   
PADX on bad circuit      : 0         
PPPoE session counters:
---------------------------------------------------------------
session down cplt recv   : 2000 session down cplt proc   : 2000
stale entry cleanup      : 0    bad state entry cleanup  : 0   
session down sent        : 0 

4.8   Step 4: Verifying PPPoE Circuit Counters

Use the show circuit counters pppoe command to check the status of the PPPoE. If the sent and received packets information is different than expected, use the show circuit counter pppoe detail command on the circuit you want to examine.

When troubleshooting PPPoE circuits, we recommend that you use the live keyword to obtain the most current information on the PPPoE circuit counters.

[local]Redback# show circuit counters pppoe live
Circuit            Packets/Bytes Sent  Packets/Bytes Received
2/2 vlan-id 100             32765              3034
                            1506597            1820700
2/2 vlan-id 100 pppoe 1     11                 10
                            480                428
2/2 vlan-id 100 pppoe 2     11                 10
                            480                428
2/2 vlan-id 100 pppoe 3     11                 10
                            480                428
... 

Use the show circuit counters pppoe detail command to display detailed information about PPPoE circuits. Verify that the expected result is displayed in the Rx, Tx PPPoE PAD counters. Check for unknown encapsulation packets counters.

The following example shows you how to check the PPP state. The PPPoE Cntrl counters, which are highlighted in bold, should increase. The remaining counters should be zero (0).

[local]Redback# show circuit counters pppoe detail
please wait... Circuit: 3/1 pppoe 1, Internal id: 1/1/7015, Encap:
ethernet-pppoe-ppp-combined

Packets                              Bytes
---------------------------------------------------------------------
Receive          :                9  Receive         :            540
Receive/Second   :             0.00  Receive/Second  :           0.00
Transmit         :               10  Transmit        :            425
Transmit/Second  :             0.00  Transmit/Second :           0.00
IP Multicast Rcv :                0  IP Multicast Rcv:              0
IP Multicast Tx  :                0  IP Multicast Tx :              0
Unknown Encaps   :                0  Unknown Encaps  : 0
Down Drops       :                0  Down Drops      :              0
Unreach Drops    :                0  Unreach Drops   :              0
Adj Drops        :                0  Adj Drops       :              0
WRED Drops Total :                0  WRED Drops Total:              0
Tail Drops Total :                0  Tail Drops Total:              0
IP Counters
Soft GRE MPLS    :                0  Soft GRE MPLS   :              0
Not IPv4 drops   :                0  Not IPv4 drops  :              0
Unhandled IP Opt :                0
Bad IP Length    :                0
Bad IP Checksum  :                0
Broadcast Drops  :                0
PPP Counters
Cntrl Rcv        :                7  Cntrl Rcv       :            277
Cntrl Tx         :                0  Cntrl Tx        :              0
Cntrl Drops Rcv  :                0
Retries Rcv      :                0
Termreqs Rcv     :                0
PPPoE Counters
Cntrl            :                2  Cntrl           :           120
Session Drops    :                0
PADT Sent        :                0
PADR Drops       :                0
PADI Drops       :                0
PADT Drops       :                0
Bad Code         :                0

4.9   Step 5: Checking the PPPoE State on the Circuit

Use the show circuit detail or show circuit handle handle detail commands to display detailed information on a circuit or circuit handle. Do the following:

• Verify that the circuit is up. Verify that the MTU size is 1492
• Verify that the PPP LCP and the PPP ICP states are Opened.
• Verify the encapsulation type is *-ethernet-pppoe-ppp-combined.

[isp]Redback# show circuit detail
Circuit: 2/1 pppoe 1, internal id: 1/1/7015, state: Up
interface bound : pool@test1
subscriber bound : pppoeoa@isp.net
bind type : chap pap
admin state : 1 hardware address : 00:30:88:00:77:0c
media type : ethernet encap type : ethernet-pppoe-ppp-combinedmtu size : 1492 cfg mtu size : 1500
ipv6 mtu size : 1500 ipv6 cfg mtu size : 1500
cct speed : 100000 cct rx speed : 0
cct flags (attr) : 0x1
slot mask : 0x0 ppa cct clear : FALSE
if flags : 0x0
profile id : 0 version : 207874
PPP OSINLCP State : Initial PPP MPLSCP State : InitialPPPOE State : READY
internal handle : 3/1:1023:63/1/1/7015

Recommended Action:

1. Check the configuration on both endpoints and make sure they match.
2. Ping the ports to check for connectivity.
3. If they are up, begin debugging at the circuit level.

For information about PPPoE debug commands, see Debugging PPPoE.

4.10   Step 6: Checking Interfaces

Use the show ip interface brief command to check that your interfaces are up. When PPPoE subscriber interface have no active subscribers, the interface is shown as down.

[isp]Redback#show ip interface briefMon Jun 27 06:38:05 2009
Name      Address            MTU   State    Bindings
1/1       4.2.13.4/16        1500   Up      ATM 1/1
2/1       10.13.49.166/24    1500   Up      ethernet 2/1
2/2       10.13.49.167/24    1500   Up      ethernet 5/1
pool      160.1.1.1/16          0   Up              

//pool indicates a multibind interface.

Recommended Action: If the interfaces are down, see Verifying Interfaces and Checking Port Performance.

4.11   Step 7: Checking Bindings

Use the show bindings command to check the configured bindings for one or more subscribers, ports, channels, or PVCs on the system. For information about this command, see Checking Bindings.

4.12   Step 8: Checking Subscribers

Use the show subscribers commands to check your subscribers. The following are some examples on how to use show subscribers commands:

For more information about how to use the show subscribers command, see Displaying Information About My Subscribers.

4.13   Step 9: Displaying PPPoE Summary Information

Use the show pppoe summary command to display information that summarizes the PPPoE sessions from the circuits of ATM, dot1q, and Ethernet. You can also use the show pppoe down command to specify that only PPP sessions for which the LCP is in the INITIAL, CLOSED, or STOPPED state are to be displayed.

[local]Redback#show pppoe summary
NUMBER
CIRCUIT TYPE   CIRCUIT   UP    DOWN
------------   ------- ------ ------
ATM            3         3     0
ETHERNET       0         0     0
DOT1Q          0         0     0
Total circuits: 3 up: 3 down: 0

Recommended Action: If your circuit is down, use the show circuit pppoe down detail command to display detailed information about PPPoE circuits that are down.

Use the show circuit pppoe detail command to display detailed information on all PPPoE circuits.

[isp]Redback#show circuit pppoe detail
Circuit: 5/1 vlan-id 108, internal id: 1/2/35, state: Down
---------------------------------------------------------------------
interface bound :
subscriber bound:
bind type       :
admin state     : 1        hardware address : 00:30:88:00:33:65
media type      : ethernet encap type       : ether-dot1q-pppoe
mode type       : 0x2      port type        : ethernet
mtu size        : 1500     cfg mtu size     : 1500
ipv6 mtu size   : 1500     ipv6 cfg mtu size: 1500
cct speed       : 100000   cct rx speed     : 0
cct flags (attr): 0x0
slot mask       : 0x0     parent slot mask  : 0x0 ppa cct clear:FALSE
if flags        : 0x0     aaa index         : 0x0
profile id      : 0       version           : 188
h node id       : 0
lg_id           : 0       spg_id            : 0
PPPOE State     : DOWN
internal handle : 5/1:1023:63/1/2/35
Summary:
   total: 1
      up: 0             down: 1
   bound: 0          unbound: 1
    auth: 0        interface: 0       subscriber: 0         bypass: 0
    no-bind: 1           atm: 0            chdlc: 0         dot1q: 0
      ether: 0            fr: 0              gre: 0
       mpls: 0           ppp: 0            pppoe: 1
      clips: 0          vpls: 0             ipip: 0
      ipsec: 0    ipv6v4-man: 0      ipv6v4-auto: 0

[local]Redback#show circuit pppoe down detail
Circuit: 5/1 vlan-id 108, internal id: 1/2/35, state: Down 
interface bound :
subscriber bound:
bind type       :
admin state     : 1          hardware address  :
00:30:88:00:33:65
media type      : ethernet   encap type        :
ether-dot1q-pppoe
mode type       : 0x2    port type        : ethernet
mtu size        : 1500   cfg mtu size     : 1500
ipv6 mtu size   : 1500   ipv6 cfg mtu size: 1500
cct speed       : 100000 cct rx speed     : 0
cct flags(attr) : 0x0
slot mask       : 0x0    parent slot mask : 0x0 ppa cct clear:FALSE
if flags        : 0x0    aaa index        : 0x0
profile id      : 0      version          : 188
h node id       : 0
lg_id           : 0      spg_id            : 0
PPPOE State     : DOWN
internal handle : 5/1:1023:63/1/2/35
Summary:
total: 1
up: 0             down: 1
bound: 0          unbound: 1
auth: 0        interface: 0       subscriber: 0           bypass: 0
no-bind: 1           atm: 0            chdlc: 0           dot1q: 0
ether: 0            fr: 0              gre: 0
mpls: 0              ppp: 0         pppoe: 1
clips: 0             vpls: 0          ipip: 0
ipsec: 0       ipv6v4-man: 0      ipv6v4-auto: 0

Recommended Action:

1. Check the configuration on both endpoints and make sure they match.
2. Ping the ports to check for connectivity.
3. If they are up, begin debugging at the circuit level.

For information about PPPoE debug commands, see Debugging PPPoE.

4.14   Step 10: Checking the PPPoE Configuration

In Table 17 is a configuration checklist for troubleshooting configuration mismatches for PPPoE subscribers. Use the show configuration pppoe command to check the PPPoE configuration.

[local]Redback#show configuration pppoe

4.15   Step 11: Checking PPPoE Process

Use the show pppoe process command to verify that the PPPoE process is running. For detailed information, use the detail keyword. If the PPPoE process is not running, use the show crashfiles command to check if there is a core dump. If there is, contact your local technical support representative.

[isp]Redback# show process pppoe
NAME   PID  SPAWN MEMORY TIME        %CPU  STATE UP/DOWN
pppoe  166  3     3436K  00:00:00.59 0.00% run   00:02:20

[isp]Redback# show crashfiles
4844 Jul 4 16:02 /md/pppoed_43.mini.core
5944456 Jul 4 16:02 /md/pppoed_43.core
4812 Jul 4 16:03 /md/pppoed_526.mini.core
5923992 Jul 4 16:03 /md/pppoed_526.core

4.16   Step 12: Displaying Subscriber Routes

Use the show ip route subscriber command to check where the subscriber is terminated. Make sure the IP address negotiated during the IPCP stage is correctly installed in the routing table.

[isp]Redback#show ip route subscriber
Codes: C - connected, S - static, S dv - dvsr, R - RIP, e B - EBGP, i B
 - IBGP
       A,H - derived hidden
       O   - OSPF, IA - OSPF inter area, N1  - OSPF NSSA external type 1
       N2  - OSPF NSSA external type 2,  E1  - OSPF external type 1
       E2  - OSPF external type 2
       i   - IS-IS, L1 - IS-IS level-1,  L2  - IS-IS level-2
       IPH - IP Host, SUB A - Subscriber address, SUB S - Subscriber
 static
       A - Derived Default
       >   - Active Route
Type    Network       Next Hop   Dist  Metric   UpTime     Interface
> SUB A 20.1.1.1/32               15      0     00:01:08   pppox
> SUB A 20.1.1.2/32               15      0     00:01:08   pppox
> SUB A 30.1.1.0/32               15      0     00:01:08   pppox

4.17   Step 13: Checking the RADIUS Server

If you have configured a RADIUS server, make sure the RADIUS attributes are configured correctly for your clients. For information about checking the RADIUS Server, see Section 16, Troubleshooting the RADIUS Server.

4.18   Step 14: Debugging PPPoE

Use the debug pppoe command to enable the generation of debug messages for various types of PPP events on the system.

Note:  

The SmartEdge 100 router does not support the standby and switchover keywords.

To store debug messages in the system log buffer, use the logging debug command (in global configuration mode). Use the show log command (in exec mode) to display these stored debug messages.

To display messages in real time, use the logging console command (in context configuration mode) if you are connected to the system through the console port. Or, use the terminal monitor command (in exec mode) if you are connected to the system through a Telnet or Secure Shell (SSH) session.

Note:  

For more information about logging commands and the terminal monitor command, see the Command List.

Use the no debug command to disable the generation of debug messages.

The following example searched for a particular host and saves the output to a file called test:

[ericsson]Redback#show subscribers log | grep 'user1@ericsson' | save test        Username = user1@ericsson,
        Username = user1@ericsson,
        Username = user1@ericsson,
        Username = user1@ericsson,
        Username = user1@ericsson,
[ericsson]Redback#cd /flash
Current directory is now /flash
[ericsson2]Redback#dir test
Contents of /flash/test
-rw-r--r--  1 root  0  135 Jun 26 19:25 /flash/test

The following example shows how to verify the contents of the test file:

[ericsson]Redback#more /flash/test
        Username = user1@ericsson,
        Username = user1@ericsson,
        Username = user1@ericsson,
        Username = user1@ericsson,
        Username = user1@ericsson,
[ericsson]Redback#

4.18.1   Using the Debug PPPoE Exception Command

Use the debug pppoe exception command to identify any unexpected events with PPPoE; for example, when a PPPoE daemon drops a packet because a session is restarting.

The following lists the different types of PPPoE exceptions:

• Dropping packet—The circuit is down (transmit path).

  Recommended Action: Check the circuit state and make sure the port or circuit is not shut down and that the link is working by using the show configuration pppoe and show port detail commands.

• Packet received on uninitialized circuit; dropped (receive path).

  Recommended Action: Check the circuit state and make sure the port or circuit is not shut down and that the link is working.

• Packet received on non-subscriber circuit; dropped—The circuit configuration is incomplete. For example, the no bind auth line or the card does not have a MAC address configured.

  Recommended Action: Check the circuit configuration by using the show configuration pppoe command.

• Session is going down; dropped—A packet got received while the session is in the process of being terminated. If it persists, an event might have been dropped due to a process restarting, such as ISM, PPP, or AAA.

  Recommended Action: Check the output and monitor recovery of that process.

• max sessions N reached— The maximum capacity on the circuit has been reached. The SmartEdge router knows how many sessions you can establish on a given circuit with the bind auth pap max nn statement, where nn can be up to 8000. When that number is reached on the circuit, an error message is displayed.
• Dropping PADR, circuit with same MAC is starting—This message mostly applies to settings of max-session > 1. The PPPoE daemon has started bringing up a session with the same MAC address, and unless that session is up, no new session with the same MAC address is accepted.

[isp]Redback# debug pppoe exception
[isp]Redback#terminal monitor
[isp]Redback##show debug // Display enabled debug commands. 

4.18.2   PPPoE Circuit Level Debug Commands

The following example show how to enable the generation of all PPPoE debug messages for all circuits on port 1 on the traffic card in slot 3:

[isp]Redback#debug circuit handle 14/1:1023:63/2/2/1
[isp]Redback#debug circuit pppoe packet

The following example show how to enable the generation of all PPPoE debug messages on circuit handle 14/1:1023:63/2/2/1:

[isp]Redback#debug circuit handle 14/1:1023:63/2/2/1
[isp]Redback#debug circuit pppoe all[isp]Redback#show debug circuit  //Shows circuits that are being debugged.
Circuit debugging:                   
14/1:1023:63/2/2/1

5   L2TP

This section describes L2TP troubleshooting.

5.1   Terminology

The following are tunnel setup control messages:

• SCCRQ (Start-Control-Connection-Request)—Initializes a tunnel between an L2TP network server (LNS) and an L2TP access concentrator (LAC). It is sent by the LAC or the LNS during the tunnel establishment process.
• SCCRP (Start-Control-Connection-Reply)—Sent in reply to a received SCCRQ message. SCCRP is used to indicate that the SCCRQ was accepted and that establishment of the tunnel should continue.
• SCCCN (Start-Control-Connection-Connected)—Sent in reply to an SCCRP. SCCCN completes the tunnel establishment process.
• HELLO—L2TP keepalive mechanism.
• StopCCN (Stop-Control-Connection-Notification)—Sent by the LAC or LNS to notify its peer that the tunnel is being shut down and the control connection should be closed. All active sessions are cleared. The reason for this request is indicated in the Result Code AVP.

The following are session (call) setup control messages:

• ICRQ (Inbound-Call-Request)—Sent by the LAC to the LNS when an incoming call is detected.
• ICRP (Inbound-Call-Reply)—Sent by the LNS to the LAC in response to a received ICRQ message
• ICCN (Inbound-Call-Connected)—Sent by the LAC to the LNS in response to a received ICRP message.
• AVP (Attribute value pair)—Format for data that is sent inside L2TP control messages.
• Active Initiator—The peer sending the SCCRQ to establish the tunnel.
• Passive Initiator—The peer receiving the SCCRQ to establish the tunnel.
• CDN—Call-Disconnect-Notify— The LAC or LNS sends a Call-Disconnect-Notify (CDN) and notifies the LNS of the disconnection of the session. The CDN contains an AVP 1 result code, which has "Loss of carrier" as the reason for the disconnect.
• Local Abort—The local system decided to close the session because it cleared the subscriber, authentication failed, or as a result of a PPP idle timeout. When a session has been established, the SmartEdge router sends a CDN to the peer.
• Remote Abort—Received a CDN from the peer. If the peer provided a reason for the disconnect, you can display it in debug output.

  If a CDN is received on the LAC for a tunnel in a tunnel group, the tunnel group load-balancing algorithm marks the tunnel as unavailable (for example, because the LNS is busy or out of resource) and only periodically tries to set up requests.

5.2   Overview

The SmartEdge router functions as an L2TP access concentrator (LAC) or as an L2TP network server (LNS). In each context configured on the system, the SmartEdge router can function as a LAC to one or more LNSs, as an LNS to one or more LACs, or as both a LAC and an LNS. LNSs and LACs are collectively referred to as L2TP peers.

With the L2TP protocol, PPP and Layer 2 endpoints can reside in different devices. PPP users are connected to the LAC, and the LAC tunnels PPP sessions to the LNS. As a result, PPP sessions are processed in a central location (LNS), regardless of where PPP users are connected.

5.3   L2TP Tunnels and Peers

The router is designated as an LNS, LTS (tunnel switch), or a LAC, depending on its tunnel function: aggregating or switching. Subscriber sessions are tunneled from the LAC to the LNS through an optional LTS :

• When functioning as an LNS, the SmartEdge router accepts sessions from LACs in the network and can either terminate them or switch them to another LNS.
• When functioning as a LAC, the SmartEdge router tunnels subscriber PPP sessions to a number of LNSs.

5.4   Tunnel Switching

The SmartEdge OS can also act as an L2TP tunnel switch (LTS), accepting PPP sessions over one tunnel and relaying them to other LNSs over another tunnel. A tunnel switch has aspects of both LAC and LNS operation.

Figure 13 shows two LACs (lac1.com and lac2.com) feeding into a tunnel switch (switch.com), which provides upstream connectivity to each indicated LNS (lns1.net and lns2.net). Here, it is assumed that the two LACs are configured to tunnel appropriate PPP sessions (perhaps all of them) to switch.com. It is also assumed that each LNS is configured to accept an L2TP tunnel to the tunnel switch.

A tunnel switch can either consult RADIUS to determine the mapping of sessions to outgoing tunnels, or it rely on the fully qualified domain name of the subscriber. In the following example, sessions with domain names of lns1.net and lns2.net are tunneled from lac1. The tunnel switch then (when not consulting RADIUS) looks for a tunnel supporting these domains. In the simplest case, if the peer name is lns1.net, the tunnel switch switches lns1.net sessions to this tunnel. With lns1.net and lns2.net, sessions coming from lac1, the tunnel switch maps each to the outgoing tunnel.

Likewise, for lns1.net and lns2.net sessions coming from lac2, the tunnel switch maps the sessions to each of the outgoing tunnels.

5.5   L2TP Packet Exchange

The following diagram describes the L2TP packet exchange.

5.6   Understanding Tunnel Information on the LAC

The following illustration describes the tunnel information on the LAC when you issue the show subscribers active command.

6   Troubleshooting Specific L2TP Issues

Before you get started, check Table 18 to determine if your have a specific L2TP issue. If so, use the associated procedure in this table as a guide to troubleshooting your specific issue.

If your issue is not listed in this section, go to the Troubleshooting General LAC Issues or Troubleshooting General LNS Issues sections.

Before you begin, get a description of the problem and check if you made any recent changes or upgrades to the network.

Note:  

When you troubleshoot specific L2TP issues, make sure you navigate to the correct context.

6.1   No Replies to SCCRQ

Symptom:

There are no replies to the SCCRQ (a control message that initializes a tunnel between an LNS and an LAC) or the SCCRQ reaches the maximum retransmits.

Cause:

To troubleshoot no replies to the SCCRQ message:

1. Ping the configured remote address (the LNS) using the source address configured. For example:

local]Redback#l2tp-peer name lns media udp-ip remote ip 50.0.0.2 local 50.0.0.1
local-name chopin
function lac-only
[local]Redback#ping 50.0.0.2 source 50.0.0.1
PING 50.0.0.2 (50.0.0.2): source 50.0.0.1, 36 data bytes,
timeout is 1 second
!!!!!

2. Use the l2tp admin test command to verify that the peer receives the SCCRQ and replies with SCCRP. For examples on how to interpret results, see Step 8: Testing L2TP Peers.
3. If the peer does not respond, issue the debug l2tp tun-setup command to enable debug messages for L2TP-related tunnel setup events:

   	Caution!
   Risk of performance loss. Enabling the generation of debug messages can severely affect system performance. To reduce the risk, exercise caution before enabling the generation of any debug messages on a production system.

[lns1]Redback#debug l2tp tun-setup

6.2   Basic L2TP Tunnel Setup Problems

Symptom:

L2TP tunnel setup problems when you receive StopCCN tunnel setup control messages or a SCCRP time-out message.

Cause:

To troubleshoot L2TP setup problems:

1. Use the l2tp admin test peer name tun-setup command. For an example about how to interpret the results of this test, see Step 8: Testing L2TP Peers
2. Use the debug l2tp all command. The system that sends the StopCCN tunnel setup control message should provide a reason for it in the debug output or in the result code AVP in the Stop CCN message.

   	Caution!
   Risk of performance loss. Enabling the generation of debug messages can severely affect system performance. To reduce the risk, exercise caution before enabling the generation of any debug messages on a production system.

6.3   Tunnel Authorization Problems

Symptom:

Tunnel authorization problems when you receive the receive StopCCN control message instead of SCCRP (active initiator) message, a StopCCN control message instead of SCCCN (passive initiator) control message, or the RADIUS Tunnel-Server-Auth-Id attribute that does not match the peer hostname AVP.

The StopCCN control message indicates that the tunnel was not established. The SCCCN completes the tunnel establishment process.

Cause:

To troubleshoot tunnel authorization problems:

1. Use the debug l2tp tun-setup command.

   	Caution!
   Risk of performance loss. Enabling the generation of debug messages can severely affect system performance. To reduce the risk, exercise caution before enabling the generation of any debug messages on a production system.

2. Use the l2tp admin test peer peer-name tun-setup command. In the output, look for the end of the tunnel that sends the StopCCN tunnel control message. The test determines why it is sending this message.
3. Use the debug l2tp avp command and verify that both peers or neither peer sends a challenge attribute value pair (AVP) .
4. If the Tunnel-Server-Auth-Id parameter is supplied, the hostname AVP sent by the peer must match this attribute value. Compare the hostname AVP in output when you issue the debug l2tp avp command and the RADIUS record.

6.4   LAC 1-Pass RADIUS Problems

Symptom:

One or more peers out of a RADIUS set are never created, which might be caused by a missing Tunnel-Server-Endpoint attribute on any tag. This is different than being created and marked dead, which is associated with a SCCRQ timeout. If the peer can be seen in the output of the show l2tp peer command, this section does not apply.

Cause: Mistakes in RADIUS tagging.

To troubleshoot LAC 1-Pass Problems:

1. Issue the debug aaa rad-packet command to enable generation of RADIUS packet debug messages.
2. Check the RADIUS response for the appropriate tagging. RADIUS debugging is useful if there is no access to the user configuration in RADIUS (or it is not a regular RADIUS server).
3. For each RADIUS tag, make sure that the appropriate Tunnel-Server-Endpoint, Tunnel-Assignment-ID, and Tunnel-Client-Endpoint attributes are present.

   The following example shows a working 1-Pass RADIUS record:

wrr-5-0-100@local, Password = "user-password"
Service-Type = Framed-User,     /* usually required by RADIUS */
Tunnel-Preference:1 = 1,        /* optional peer preference */
Tunnel-Server-Endpoint:1 = "42.1.0.5", /* LNS's address */
Tunnel-Client-Endpoint:1 = "81.5.1.189", /* local addr on LAC */
Tunnel-Client-Auth-Id:1 = "lac01",    /* local-name on LAC */
Tunnel-Assignment-Id:1 = "primary", /* optional assignmentId */
Tunnel-Preference:2 = 2,
Tunnel-Server-Endpoint:2 = "42.1.0.6",
Tunnel-Client-Endpoint:2 = "81.5.1.189",
Tunnel-Client-Auth-Id:2 = "lac01",
Tunnel-Assignment-Id:2 = "backup

4. In the debug output, look for StopCCN control messages and check the Result-Code AVP for any messages indicating why you have LAC1-Pass problems. The StopCCN message indicates that the L2TP tunnel was not established.
5. Use the debug ses-abort and debug tun-setup to determine if the remote system was closed by administrator.

   	Caution!
   Risk of performance loss. Enabling the generation of debug messages can severely affect system performance. To reduce the risk, exercise caution before enabling the generation of any debug messages on a production system.

6.5   L2TP Session Setup

Symptom:

If you are not receiving an ICRP control message from a LNS, or if the LAC receives the CDN from the LNS instead of the ICRP, you might have l2TP session setup issues.

Causes:

To troubleshoot L2TP session setup issues:

1. Use the ping and traceroute commands to check the connectivity on the L2TP tunnel.
2. Use the debug l2tp ses-abort command on the LAC and LNS. The sender of the CDN should have a debug message before sending the Call-Disconnect-Notify (CDN) message, and the CDN may have a result-code AVP with a disconnect reason message (or ppp-disconnect cause or Redback term cause). The Redback term causes are termination error codes (codes that identify what caused the error or what caused the subscriber to go down). They are in the L2TP disconnect messages (such as CDN, StopCCN), and are specific to Redback Networks.

   	Caution!
   Risk of performance loss. Enabling the generation of debug messages can severely affect system performance. To reduce the risk, exercise caution before enabling the generation of any debug messages on a production system.

3. Use the l2tp admin test peer peer-name ses-setup [count number] to see if the LNS sends an ICRP. If the control channel is operating poorly, it can adversely impact session setups because of timing issues (and concurrency).
4. Check if the first packet from LNS is being dropped.

6.6   CDN for an Unknown Session ID or Duplicate Remote Session ID

Symptom: A CDN for an unknown session ID or a duplicate remote session ID (receiving an ICRQ).

Cause: Problems with peer software.

To troubleshoot a CDN for an unknown session ID or a duplicate remote session ID (receiving an ICRQ):

1. Use the debug l2tp packet command to enable debug messages for L2TP-related packet transmit (TX) and receive (RX) events for L2TP control messages.

   	Caution!
   Risk of performance loss. Enabling the generation of debug messages can severely affect system performance. To reduce the risk, exercise caution before enabling the generation of any debug messages on a production system.

2. Use the debug l2tp AVP command to enable debug messages on L2TP attribute-value pairs (AVPs) transmitted or received in L2TP control messages.

6.7   Stuck Sessions or Tunnels

Symptom: Session count is greater than 0 and no subscribers are up.

Recommended Action: On the LAC, perform the tasks listed at Tasks to Troubleshoot General LAC Issues. On the LNS, perform the tasks list at Tasks to Troubleshooting General LNS Issues.

6.8   Scaling Issues

Symptom:

You might have scaling issues if:

• The ICRQs are sent for sessions that have recently been shut down.
• There are FSM timeouts—The session is stuck in wait-ICRP on the LAC or wait-ICCN on the LNS.
• There is no response to the ICRQ, ICRP, or ICCN.
• On the LAC, you receive a Receive Term-Req from client before the ICRP from the LNS.

Cause:

The control channel might be operating too slowly.

To troubleshooting L2TP scaling issues:

1. Trace the session setup using the l2tp admin test peer name ses-setup count number command. Look at the output to determine how to address any issues.
2. If the tunnel remains up (it does not terminate from maximum retransmits), but symptoms are present, check the session setup debug output on the remote system.
3. Use the debug l2tp ses-abort command to check if the client sends a Term-Req attribute before the ICRP or if there are finite state machine (FSM) time-outs.

   	Caution!
   Risk of performance loss. Enabling the generation of debug messages can severely affect system performance. To reduce the risk, exercise caution before enabling the generation of any debug messages on a production system.

4. On the LAC, use the l2tp admin test command to trace a session setup and see if the LNS is sending the ICRP.
5. If the results for the test indicate that the control channel might be operating too slowly, use the l2tp admin down peer name seconds seconds command for several seconds to see if it fixes this issue. This command marks the L2TP peer as “dead”; no new sessions are assigned to this peer. If the control window is running poorly, this can help clean out the queue.

6.9   Fragmentation

Symptom:

If your subscribers can connect and log on, but certain sites are not accessible, you might have a fragmentation issue.

Cause:

The client TCP maximum segment size (MSS) is set too high for L2TP tunneling, forcing incorrect fragmentation or no fragmentation.

To troubleshoot fragmentation issues:

1. Check the maximum received unit (MRU) values negotiated with the client on the LAC and LNS by using the ppp debug packet command. Look for negotiated fragmentation mismatches.

   	Caution!
   Risk of performance loss. Enabling the generation of debug messages can severely affect system performance. To reduce the risk, exercise caution before enabling the generation of any debug messages on a production system.

2. On the client, check that the appropriate TCP MSS is set.
3. Check the TAC FAQ for fragmentation issues.

6.10   LNS PPP Session Setup

Symptoms:

• The LNS sends the LCP Term-Req attribute after renegotiating the LCP.
• The client sends the LCP Term-Req attribute instead of renegotiating LCP.
• The call disconnects soon after the ICCN message is sent by the LAC.

Cause:

To troubleshooting LNS PPP session setup problems:

1. Use the debug ppp lcp command to check the session setup on the LNS.
2. Troubleshoot AAA by using the commands listed in Troubleshooting the RADIUS Server.
3. Use the show l2tp configuration command to verify that the session authentication configuration on the LNS matches the authentication protocols available on the client. Make sure the LAC bind authentication protocols agree with the LNS session authentication configuration protocols.
4. Use the show configuration l2tp command and check for l2TP configuration mismatches in the MRU negotiation configuration (configured by using the ppp peer-options mru command in global configuration mode) and the LT2P renegotiation configuration on the LNS.
5. Use the debug ppp packet command on the LAC and LNS.

Recommended Actions:

• If you have enabled the l2tp renegotiate lcp on-mismatch command, change the value of the l2tp renegotiate mru to agree with client if you want to avoid renegotiation.
• Force renegotiation on the LNS by using the l2tp renegotiate lcp always command.
• On the LAC, prevent it from sending proxy AVPs by issuing the no l2tp proxy-auth command.

7   Troubleshooting General LAC L2TP Issues

This section describes how to troubleshoot general LAC L2TP Issues. Before you begin, get a description of the problem and check if you made any recent changes or upgrades to the network. For information about the operational commands, see the Command List.

The following is a sample configuration for a SmartEdge router that is acting as a LAC (a LNS peer), in context LAC1. The following configuration maps to the tasks listed in the Tasks to Troubleshoot LAC General L2TP Issues section.

context LAC1
 domain isp1.net                   //Assign a domain alias for this context. 
 interface loop1 loopback
  ip address 1.1.1.1/32
 interface toLNS1
  ip address 192.168.6.2/30
 ip route 0.0.0.0/0 192.168.6.1
 l2tp-peer name LNS1 media udp-ip remote ip 1.1.1.2 local 1.1.1.1
  session-auth pap
  function lac-only               //Specify the role as a LAC.  
  local-name LAC1                 //Name of LAC
  aaa authentication subscriber local //This line is normally hidden because 
 subscriber name pppoe1               //it is the default behavior. 
  password test3
  tunnel name LNS1               //Static peer selection for subscriber sessions.
!
atm profile ubr 
 shaping ubr 
!
port atm 1/1
 no shutdown
 atm pvc 1 32 profile ubr encapsulation pppoe 
  bind authentication pap context LAC1
!
port eth 5/1
 no shutdown
 encap dot1q
 dot1 pvc 100
  bind interface toLNS1 LAC1

7.1   Tasks to Troubleshoot General LAC L2TP Issues

Use the following table as a guide to troubleshooting general L2TP issues on the LAC.

Note:  

When working with technical support representatives, use the show tech-support command on your router and have the output available for technical support representatives to assist in troubleshooting.

7.2   Step 1: Checking Specific LAC Issues

Check if you have a specific LAC issue listed in the Troubleshooting Specific L2TP Issues section. If you do, use the procedure in this section to resolve your issue. If not, go to step 2.

7.3   Step 2: Navigating to the Correct Context

Use the show context all command and display all the contexts on your router and then navigate to the context you want to troubleshoot, in this case, the LAC1 context.

The following example shows how to view all contexts on your router and then navigate to context LAC1:

[local]Redback#show context allContext Name          Context ID       VPN-RD   Description
-----------------------------------------------------------
local                 0x40080001
LAC1                  0x40080002

[local]Redback#
[local]Redback#context LAC1
[LAC1]Redback#

7.4   Step 3: Checking LAC Port Counters

Use the show port counters live command to check the port counters on the LAC in real time. Use the detail keyword to display detailed port counter information. For more information about this command, see Checking Port Performance.

[LAC1]Redback#show port counters live
please wait...
Port            Type
1/1             ATM
packets sent       : 15000             bytes sent    : 20000
packets recvd      : 10000             bytes recvd   : 50000
send packet rate   : 0.00              send bit rate : 0.00
recv packet rate   : 0.00              recv bit rate  : 0.00
rate refresh interval : 60 seconds
5/1           ethernet
packets sent       : 25000             bytes sent    : 40000
packets recvd      : 60000             bytes recvd   : 80000
send packet rate   : 0.00              send bit rate : 0.00
recv packet rate   : 0.00              recv bit rate : 0.00
rate refresh interval : 60 seconds

7.5   Step 4: Displaying LAC Tunnel-Level Counters

Use the show l2tp counters peer command to display all statistics for tunnel-level control packet counters for all tunnels under specific L2TP peers on the system.

Use the tunnel tunl-id construct to display counters for a specific L2TP tunnel.

Use the detail keyword to display detailed information for each L2TP counter. Use the optional tunnel tunl-id construct to display all statistics for tunnel-level control packet counters for all tunnels with a specific tunnel ID.

The following example show you how to display detailed information for counters for L2TP tunnel 28561:

[LAC1]Redback#show l2tp counters peer LAC1 tunnel 28561 detail
Local ID:            28561      Remote ID:           30416
Local IP:            1.1.1.1    Remote IP:           1.1.1.2
Local Name:          LAC1       Remote Name:         LNS1
Session Count:       101        Active Sessions:     100
Total Est Sessions:  120        Total Fail Sessions: 2
Max Sessions Ever:   100
Uptime :             50 mins 2 secs

Control Statistics
Tx Control Packets:  165031     Rx Control Packets:  165031
Tx Control Bytes:    2862224    Rx Control Bytes:    3015284020
Tx Hello Packets:    234        Rx Hello Packets:    235
Ns:                  2445       Nr:                  2457
Tx Cwnd:             3          Remote Window Size:  10
Resend Q Size:       8          Unsent Q Size:       6
Max Resend Q Size:   8          Max Unsent Q Size:   8
Control Errors:      2
Control Message Times
Last Msg sent:       Tue 16:34:06 (9 mins 19 secs ago)
Last Msg recved:     Tue 16:34:06 (9 mins 19 secs ago)
Last Hello sent:     Tue 16:34:06 (9 mins 19 secs ago)
Last Hello rcvd:     Tue 16:34:06 (9 mins 19 secs ago)
Last Control Error:  Tue 16:34:06 (9 mins 19 secs ago)
Data Statistics
Tx Data Packets:     4              Rx Data Packets:     3
Tx Data Bytes:       43             Rx Data Bytes:       36

7.6   Step 5: Displaying LAC IPC Counters

Use the show l2tp global group-name command to view IPC counters for all L2TP peers and groups on the LAC.

The following example displays the IPC counters for 16,000 subscriber circuits on each of two traffic cards in slots 12 and 13:

[LAC1]Redback#show l2tp global ipc
L2TP IPC Stats

Up since:............Fri 08:13:05 (3 hours 30 mins 25 secs ago)
Last clock tick:.....Fri 11:43:30 (0 secs ago)

ISM MBE:
Sessions Recovered/Failed:......0/0
Tunnels Recovered/Failed:.......0/0
Idle Session Cleanups:..........0
Couldn't Allocate Msg:..........0
TX Circuit Tunnel-Start:........91867
TX Circuit Tunnel-Stop:.........0
TX Circuit Stale Kills:.........0
TX Sub-Sess-Down:...............26
Tunnel Start/Stop Errors:.......0

PPP:
RX Start request:...............0

AAA:
TX Tunnel Author:...............0
RX Tunnel Author:...............0
PPAs:
Registrations:..................12
Stale Registrations:............0
Registration Errors:............0
TX Tunnel Create:...............16179
TX Tunnel Delete:...............95
IPC Errors:.....................0

Registered PPAs:
Slot 01 02 03 04 05 06 07 08 09 10 11 12 13 14
I  I  I  I  .  .  .  .  .  .  .  I  I  .
E  E  E  E  .  .  .  .  .  .  .  E  E  .

PPA Name     Registration Time    Chg#  Ccts Tun-Cr   Tun-Del
------------ -------------------- ---- ----- -------- -------
Slot  1 IPPA Fri Nov 05 08:13:39     2     1    16179     95
Slot  1 EPPA Fri Nov 05 08:13:40     4     1    16179     95
Slot  2 IPPA Fri Nov 05 08:13:41     6     2    16179     95
Slot  2 EPPA Fri Nov 05 08:13:42    16     2    16179     95
Slot  3 IPPA Fri Nov 05 08:13:41     8     4    16179     95
Slot  3 EPPA Fri Nov 05 08:13:42    18     4    16179     95
Slot  4 IPPA Fri Nov 05 08:13:41    10     0    16179     95
Slot  4 EPPA Fri Nov 05 08:13:42    20     0    16179     95
Slot 12 IPPA Fri Nov 05 08:13:41    12 16004    16179     95
Slot 12 EPPA Fri Nov 05 08:13:42    22 16004    16179     95
Slot 13 IPPA Fri Nov 05 08:13:42    14 16002    16179     95
Slot 13 EPPA Fri Nov 05 08:13:42    24 16002    16179     95

7.7   Step 6: Checking LAC Interfaces

On the LAC, check that the interfaces are up by using the show ip interface brief command.

[LAC1]Redback#show ip interface brief
Mon Jun 27 06:38:05 2009
Name      Address             MTU    State    Bindings
1/1       4.2.13.4/16         1500   Up       ATM 
5/1       10.13.49.166/24     1500   Up       ethernet 

Recommended Action: If the interfaces are down, see Verifying Interfaces and Checking Port Performance.

7.8   Step 7: Verifying LAC L2TP Establishment

To verify LAC L2TP establishment, use the ping and traceroute commands to verify that links between the LAC and LNS are operational. If a ping between routers is not successful, the link is not functioning properly and you need to troubleshoot it. Use the source address option, because loopbacks are often used:

[LAC1]Redback#ping 1.1.1.2 source 1.1.1.3
[LAC1]Redback#traceroute 1.1.1.2

7.9   Step 8: Checking the LAC Status of L2TP Peers

On the LAC, check the status of L2TP peers by using the show l2tp peer command.

If the session count = 0, there is no PPP session. If the session count is > 1, PPP sessions have been established. If the tunnel count = 0, the tunnel is down. If the tunnel count is > 1, the tunnel has been established. In Conf. source, Local indicates a local configuration; RADIUS indicates a RADIUS configuration.

[LAC1]train-4# show l2tp peer
                                  Conf.   Tun   Ses
Peer Name   Local Name       Role Source  Count Count
----------  ---------------- ------------ ----  ----- 
LNS1        LAC1             LAC  Local   1     1
[LAC1]train-4#

[LAC1]Redback#show l2tp peer LNS1
Peer Name:        LNS1
Admin State:      Up
Local Name:       LAC1
Vendor:           RedBack Networks (Firmware: 0x0600)
Local IP Address: 1.1.1.1    Remote IP Address:  1.1.1.2
Local Role:       LAC Only   DNIS:               Disabled
Hello Timer:      300        Preference:         10
Maximum Tunnels:  32767      Maximum Ses/Tunnel: 65535
Control Timeout:  3          Retry:              10
Tunnel Count:     1          Session Count:      1
Domains: isp1.net #Peer name LNS1 is assigned domain isp1.net
[LAC1]train-4#

7.10   Step 9: Performing LAC L2TP Tunnel and Session Testing

7.10.1   Step 1: Tunnel and Session Testing

On the LAC, use the l2tp admin test peer command to perform L2TP peer testing. You can test any idle tunnels to a peer using Hello messages, test the tunnels, and test sessions on tunnels. Check for StopCCN control messages.

Use the ses-setup keyword to test if an L2TP network server (LNS) peer allows a session to be set up without a client connecting to it. The L2TP session is established, but the Point-to-Point Protocol (PPP) is not negotiated for the session with the peer; as a result, the peer times out and closes the session.

Use the tun-setup keyword to test if a tunnel can be created to a peer; this test validates the remote IP address (LNS) and, if configured, the local IP address specified by the l2tp-peer command (in context configuration mode), and the tunnel authorization key specified by the tunnel-auth key command (in L2TP peer configuration mode).

The following is an example of an unsuccessful L2TP peer test:

[LAC1]Redback#l2tp admin test peer LNS1 tun-setup
Apr 12 17:00:02: %L2TP-7-TUN: Choosing endpoints for tunnel LNS1:30947
Apr 12 17:00:03: %L2TP-7-TUN: LNS1:30947 local address dynamic:
50.0.0.2:1701/50.0.0.1:1701 
Apr 12 17:00:03: %L2TP-7-TUN: LNS1:30947 endpts cfg: 50.0.0.2:1701/50.0.0.1:1701 
Apr 12 17:00:03: %L2TP-7-TUN: Sending SCCRQ to LNS1:30947 at 50.0.0.2
Apr 12 17:00:03: %L2TP-7-TUN: LNS1:30947 Sent SCCRQ
Apr 12 17:00:03: %L2TP-7-TUN: LNS1:30947 No response from peer to our SCCRQ 
Apr 20 12:40:05: %L2TP-7-AVP:  M  Len=46 IETF Result-Code=(2,6):
Control channel timeout - Remote peer dead
Apr 12 17:00:03: %L2TP-7-TUN: Choosing endpoints for tunnel LNS1:30947
Apr 12 17:00:03: %L2TP-7-TUN: LNS1:30947 endpts cfg:50.0.0.2:1701/50.0.0.1:1701
Apr 12 17:00:03: %L2TP-7-TUN: LNS1:30947 Sending Stop SCCCN 

Recommended Action: Ping the LNS from the LAC context. Verify the IP routes using the show ip route command.

In the following example, the No response from peer to the SCCRQ message indicates that the LNS was not reachable; the Stop SCCN message indicates that the L2TP tunnel was not established.

1. Ping the LNS from the LAC context.
2. Verify the IP routes using the show ip route command.

7.10.2   Step 2: Verifying Subscriber Attributes on the LAC

On the LAC, verify subscriber attributes by using the show subscribers active command.

The following illustration describes the tunnel information on the LAC when you issue this command:

To view some examples on how to use show subscribers commands, see Troubleshooting PPP, Step 9: Checking Subscribers.

7.10.3   Step 3: Verifying Where Subscribers are Terminated on the LAC

On the LAC, verify where the subscribers are terminated by using the show subscribers all command:

[LAC1]train-4# show subscribers all
[TYPE    CIRCUIT                    SUBSCRIBER    CONTEXT  START  TIME
-------------------------------------------------------------------------
pppoe   13/1:1 vpi-vci 0 100 pppoe  user@isp3.net LAC1     Jan 4 15:25:12
-------------------------------------------------------------------------
Total=1

7.11   Step 10: Displaying L2TP Group Information on the LAC

If you have configured a L2TP group, use the show l2tp group command to view the redundancy algorithm and deadtime of one specific L2TP group or for all groups in the current context. When you display information for a specific group, the names of the peer members of the group and information about each peer are also displayed.

Note:  

The LAC1 sample configuration does not have L2TP groups.

The following example displays output from the show l2tp group command without specifying a group name:

[local]Redback#show l2tp group 
Group Name       Algorithm    Deadtime 
---------------- ------------ -------- 
l2tp             Load-balance 10
l2tp2            Load-balance 5
l2tp3            Load-balance 10

The following example displays the output when you use the show l2tp group command to display a particular group (l2tp). The asterisk (*) in front of the peer, l2tp-1, indicates that the peer is down (“dead”).

[LAC1] Redback#show l2tp group l2tp
Group name:      l2tp  RADIUS:  YES 
Algorithm        Load-balance  Deadtime: 10 
Peers:          *l2tp-1
                 l2tp-2 

Domains: vpn 
                            Max  Tun    Max   Ses 
Peer Name  Local Name Med  Tuns  Cnt    Ses   Cnt  Stat LAC  LNS  Named
--------- ----------  ---   ----  ---   ---   ---- ---- ---  ---  -----
l2tp-1     tgrp1       UDP   1     0     20    0   NO   YES  YES   YES
l2tp-2     tgrp2       UDP   1     1     65535 6   NO   YES  YES   YES

7.12   Step 11: Displaying LAC Summary Peer Information

On the LAC, use the show l2tp summary command to display information about all peers in all the contexts:

[LAC1]Redback#show l2tp summary
Context Name   Peer Name    Local Name    Tun      Ses
                                          Count    Count
-------------  ----------   ----------    -----    -----
LNS1            LAC1        LNS1            1        0
LAC1            LNS1        LAC1            1        0

7.13   Step 12: Checking LAC Bindings

On the LAC, use the show bindings summary command to verify your bindings. For information about this command, see Checking Bindings.

7.14   Step 13: Checking the LAC IP Route Summary Information

On the LAC, use the show ip route summary command to display summary information for all IP routes.

7.15   Step 14: Verifying the LAC Routes

On the LAC, use the show ip route command to display information about all IP routes or routes for only the specified IP address or IP prefix. For information about this command, see the Command List.

7.16   Step 15: Displaying LAC PPP Summary Information

On the LAC, use the show ppp summary command to display summary information for all PPP sessions in the current context: LAC1. For more information about this command, see Displaying Summary of PPP Information.

7.17   Step 16: Checking LAC L2TP Process

On the LAC, use the show process l2tp command to verify that the L2TP process is running. Use the detail keyword to view detailed information about the L2TP process. If the process is not running, use the show crashfiles command to check if there is a core dump. If there is, contact your local technical support representative.

[LAC1]Redback#show process l2tp
NAME PID  SPAWN  MEMORY TIME         %CPU   STATE  UP/DOWN
l2tp  166 3      3436K  00:00:00.59  0.00%  run    00:02:20

7.18   Step 17: Checking the LAC L2TP Configuration

On the LAC, use the show configuration l2tp command and the following table as a guide to check for common configuration mismatches.

[LAC1]Redback# show configuration l2tp

7.19   Step 18: Checking the RADIUS Server

If you have configured a RADIUS server, make sure the RADIUS attributes are configured correctly for your clients. For information about checking the RADIUS Server, see Section 16, Troubleshooting the RADIUS Server.

7.20   Step 19: Debugging the LAC

For information about debugging L2TP, see Using L2TP Debug Commands.

8   Troubleshooting LNS General L2TP Issues

This section shows you how to troubleshooting general L2TP issues.

The following is a sample LNS configuration for the SmartEdge router acting as a LNS for a LAC peer in the context LNS1.

context LNS1
  domain isp1.net            //Domain alias for the context to use the LAC peer.
    interface loop1 loopback
    ip address 1.1.1.2/32
 interface toLAC1
   ip address 192.168.6.1/30
 interface toInternet 
  ip address 2.1.1.2/24
   ip route 0.0.0.0/0 2.1.1.1.254
   ip route 1.1.1.1/32 192.168.6.2 
 l2tp-peer name LAC1 media udp-ip remote ip 1.1.1.1 local 1.1.1.2
   session-auth pap
   function lns-only         //Specify that this tunnel definition will act as a LNS. 
   local-name LNS1
!
  interface forSubs multibind
   ip address 10.1.1.1/24 
   ip pool 10.1.1.1/24 
 aaa authentication subscriber none
 subscriber default
  ip address pool
!
 port ethernet 2/1
  encap dot1q
  no shutdown
  dot1 pvc 100
      bind inter toLAC1 LNS1
!
port ethernet 2/4
 no shutdown
bind interface toInternet LNS1

8.1   Tasks to Troubleshoot General LNS Issues

Use the following table as a guide to troubleshooting general L2TP issues on the LNS. Before you begin, get a description of the problem and check if you made any recent changes or upgrades to their network.

For information about operational commands, see the Command List.

Note:  

When working with technical support representatives, use the show tech-support command on your router and have it available to your technical support representative to assist in troubleshooting.

8.2   Step 1: Checking Specific LNS Issues

Check if you have a specific LNS issue listed in the Checking Troubleshooting Specific L2TP Issues section. If you do, use the procedure in this section to resolve your issue. If not, go to step 2.

8.3   Step 2: Navigating to the Correct Context on the LNS

Use the show context all command to display all the contexts on the SmartEdge router and then navigate to the context in which you have configured the LNS—in this case, the LNS1 context.

[local]Redback#show context all
Context Name  Context ID    VPN-RD    Description
-------------------------------------------------
local         0x40080001
LNS1          0x40080002

[local]Redback#
[local]Redback#context LNS1
[LAC1]Redback#

8.4   Step 3: Checking LNS Port Counters

On the LNS, use the show port counters command to check the port counters. Use the detail keyword to displayed detailed information about the port counters. For more information about this command, see Checking Port Performance.

[LNS1]Redback#show port counters live
please wait...
Port            Type
2/1             ethernet
packets sent       : 15000      bytes sent     : 20000
packets recvd      : 10000      bytes recvd    : 50000
send packet rate   : 0.00       send bit rate  : 0.00
recv packet rate   : 0.00       recv bit rate  : 0.00
rate refresh interval : 60 seconds
2/4             ethernet
packets sent       : 25000      bytes sent     : 40000
packets recvd      : 60000      bytes recvd    : 80000
send packet rate   : 0.00       send bit rate  : 0.00
recv packet rate   : 0.00       recv bit rate  : 0.00
rate refresh interval : 60 seconds

8.5   Step 4: Checking LNS Circuit Counters

On the LNS, use the show circuit counters l2tp all command to display summary information for all circuits.

[LNS1]Redback#show circuit counters l2tp all
Circuit        Packets/Bytes Sent    Packets/Bytes Received
L2TP LNS 1  
                        0                     0
                        0                     0

On the LNS, use the show circuit counters l2tp lns circuit_ID command to display information on a specific circuit handle:

[LNS1]Redback#show circuit counters l2tp lns 1
Circuit      Packets/Bytes Sent    Packets/Bytes Received
L2TP LNS 1                               
                   10                         9
                  534                       229

8.6   Step 5: Displaying LNS Tunnel-Level Counters

On the LNS, use the show l2tp counters peer peer-name tunnel command to display tunnel-level counters for L2TP control packets.

Use the tunnel tunl-id construct to display counters for a specific L2TP tunnel. Use the detail keyword to display detailed information for each L2TP counter.

Use the optional tunnel tunl-id construct to display all statistics for tunnel-level control packet counters for all tunnels with a specific tunnel ID.

The following displays tunnel-level counters on the LNS:

[LNS1]Redback#show l2tp counter peer tr069 tunnel 
Local ID:             13127        Remote ID:   6934  
Local IP:            1.1.1.2       Remote IP     1.1.1.1
Local Name:          tr069-tunnel-client  Remote Name:  l2tp-tr069
Session Count:       4             Active Sessions:     4  
Total Est Sessions:  15            Total Fail Sessions: 2   
Max Sessions Ever:   5  
Uptime :             11 hours 25 mins 18 secs 
Control Statistics
Tx Control Packets:  191           Rx Control Packets:  173  
Tx Control Bytes:    7669          Rx Control Bytes:    3025  
Tx Hello Packets:    126           Rx Hello Packets:    0  
Ns:                  166           Nr:                  25 
Tx Cwnd:             130           Remote Window Size:  20050   
Resend Q Size:       0             Unsent Q Size:       0 
Max Resend Q Size:   2             Max Unsent Q Size:   0    
 Control Errors:     0 

8.7   Step 6: Displaying LNS IPC Counters

Use the show l2tp global group-name command to view IPC counters for all L2TP peers and groups on the LNS.

The following example displays the IPC counters for 16,000 subscriber circuits on each of two traffic cards in slots 12 and 13:

[LNS1]Redback#show l2tp global ipc
L2TP IPC Stats

Up since:............Fri 08:13:05 (3 hours 30 mins 25 secs ago)
 Last clock tick:.....Fri 11:43:30 (0 secs ago)

ISM MBE:
  Sessions Recovered/Failed:......0/0
  Tunnels Recovered/Failed:.......0/0
  Idle Session Cleanups:..........0
  Couldn't Allocate Msg:..........0
  TX Circuit Tunnel-Start:........91867
  TX Circuit Tunnel-Stop:.........0
  TX Circuit Stale Kills:.........0
  TX Sub-Sess-Down:...............26
  Tunnel Start/Stop Errors:.......0

 PPP:
  RX Start request:...............0

 AAA:
  TX Tunnel Author:...............0
  RX Tunnel Author:...............0

 PPAs:
  Registrations:..................12
  Stale Registrations:............0
  Registration Errors:............0
  TX Tunnel Create:...............16179
  TX Tunnel Delete:...............95
  IPC Errors:.....................0

  Registered PPAs:

Slot 01 02 03 04 05 06 07 08 09 10 11 12 13 14
      I  I  I  I  .  .  .  .  .  .  .  I  I  .
      E  E  E  E  .  .  .  .  .  .  .  E  E  .

 PPA Name     Registration Time   Chg# Ccts  Tun-Cr  Tun-Del
 ------------ ------------------- ---- ----- -------- ------
 Slot  1 IPPA Fri Nov 05 08:13:39    2     1    16179   95
 Slot  1 EPPA Fri Nov 05 08:13:40    4     1    16179   95
 Slot  2 IPPA Fri Nov 05 08:13:41    6     2    16179   95
 Slot  2 EPPA Fri Nov 05 08:13:42   16     2    16179   95
 Slot  3 IPPA Fri Nov 05 08:13:41    8     4    16179   95
 Slot  3 EPPA Fri Nov 05 08:13:42   18     4    16179   95
 Slot  4 IPPA Fri Nov 05 08:13:41   10     0    16179   95
 Slot  4 EPPA Fri Nov 05 08:13:42   20     0    16179   95
 Slot 12 IPPA Fri Nov 05 08:13:41   12 16004    16179   95
 Slot 12 EPPA Fri Nov 05 08:13:42   22 16004    16179   95
 Slot 13 IPPA Fri Nov 05 08:13:42   14 16002    16179   95
 Slot 13 EPPA Fri Nov 05 08:13:42   24 16002    16179   95

8.7.1   Verifying Session Mapping to the Context and Tunnel

PPP sessions are mapped to contexts in the following ways:

1. Global RADIUS is used, and the context to map the subscriber session is returned from RADIUS.
2. The fully qualified domain name is used to map a subscriber PPP session to a context. The context name as well as additional domain statements in each context are used to map the sessions.

   The global configuration statement aaa last-resort context context-name assigns sessions to the named context if no matching domain-name statement is found.

3. The circuit bind statement can force or override the domain switching logic. For example, the bind authentication pap context LAC1 command forces every session to context “LAC1”, regardless of the subscriber session domain.

Once mapped to a context, PPP sessions can be mapped to a tunnel in the following ways:

1. Based on domain; for example, the name or domain statement on the tunnel peer or peer group (by using the aaa auth subscriber none command or tunnel domain in the subscriber default).
2. By using the tunnel name tunnel_name command under the subscriber default command or under the local subscriber record.
3. By RADIUS returning tunnel peer name local authentication and tunnel assignment in the subscriber record.

8.8   Step 7: Verifying LNS L2TP Establishment

8.8.1   Step 1: Checking Connectivity

From the LNS, ping the LAC to test connectivity.

Use the ping and traceroute commands to verify that links between LNS and LAC are operational. If a ping between routers is not successful, the link is not functioning properly, and you need to troubleshoot it. Use the source address option because loopbacks are often used.

[LNS1]Redback#ping 1.1.1.1 source 1.1.1.3
[LNS1]Redback#traceroute 1.1.1.1

8.8.2   Step 2: Verify Where Subscribers are Terminated

On the LNS, use the show subscribers all command to verify where the subscribers are terminated:

[LNS1]train-4# show subscribers all
TYPE CIRCUIT           SUBSCRIBER    CONTEXT START TIME
-----------------------------------------------------------
ppp  L2TP LNS 5245747  user@isp1.net ISP3    Jan 4 15:25:12
-----------------------------------------------------------
Total=1

8.8.3   Step 3: Verifying Subscribers Attributes

On the LNS, use the show subscribers active command to make sure the subscriber attributes were applied correctly.

[LNS1]train-4#context ISP3
[ISP3]train-4#show subscribers activeuser@isp3.net
Circuit L2TP LNS 6587802
Internal Circuit 255/16:1023:63/5/2/6587802
Current port-limit unlimited
timeout idle 36000 (applied)
ip address 10.1.1.2 (applied from pool)
tunnel type 3 (applied)
tunnel medium type 1 (applied)
tunnel server endpoint 1.1.1.2 (applied)
tunnel client endpoint 1.1.1.1 (applied)
tunnel server auth ID LNS1 (applied)
tunnel client auth ID LAC1 (applied)
tunnel max sessions 65535 (applied)
tunnel max tunnels 32767 (applied)
tunnel function 2 (applied)
tunnel connection LAC1:27201:54912 (applied)
tunnel vendor avp (applied)
tunnel vendor avp (applied
[ISP3]train-4#

For more examples on how to use show subscribers commands, see Checking Subscribers in the Troubleshooting PPP section.

8.9   Step 8: Checking L2TP Peers on LNS

On the LNS, check the status of L2TP peers by using the show l2tp peer command. Check the number of tunnels, profiles, session count (number of subscribers for the tunnel) and the attributes (Auth Method, Domains, Session-Context, maximum sessions and tunnels) applied to the tunnels:

[LNS1]train-4# show l2tp peer
                                               Conf.  Tun   Ses
Peer Name              Local Name        Role  Source Count Count
--------------------   ----------------  ----  ------ ----- -----
LAC1                   LNS1              LNS   Local  1     0

[LNS1]Redback#show l2tp peer LAC1
Peer Name:             LAC1
Admin State:           Up
Local Name:            LNS1
Vendor:                RedBack Networks
Local IP Address:      1.1.1.2         Remote IP Address:  1.1.1.1
Local Role:            LNS Only        DNIS:               Disabled
Hello Timer:           300             Preference:         10
Maximum Tunnels:       32767           Maximum Ses/Tunnel: 65534
Control Timeout:       3               Retry:              10
Tunnel Count:          1               Session Count:      2
Authen Method:         CHAP-PAP        Session-Context:    <Any>
Domains: (NO DOMAINS)
[LNS1]Redback#

[LNS1]train-4# show l2tp peer
                                               Conf.  Tun   Ses
Peer Name              Local Name        Role  Source Count Count
--------------------   ----------------  ----  ------ ----- -----
LAC1                   LNS1              LNS   Local  1     0

[LNS1]train-4# show l2tp peer LAC1 tunnel
Loc    Rem    Remote          Rem    Ses   Act   Total   Total
ID     ID     IP Address      Port   Count Ses   Estab   Fail
-----  -----  --------------- ------ ----- ----- ------- -------
29048  29047  1.1.1.1         1701   0     0     1       0

[LNS1]train-4# show l2tp peer LAC1 tunnel 29048
State:                 Established
Last change:           Wed 14:42:27 (7 mins 7 secs ago)
Local ID:              29048          Remote ID:           29047
Local IP:              1.1.1.2        Remote IP:           1.1.1.1
Local Name:            LNS1           Remote Name:         LAC1
Session Count:         0              Active Sessions:     0
Total Est Sessions:    1              Total Fail Sessions: 0
Control Channel:
Window current-tx:     3 max-tx: 8 rx: 8
Forwarding:
Next-Hop Circuit:      11/2
Peer-facing Card(s):   11
Tunnel PPA Table:
Slot 01 02 03 04 05 06 07 08 09 10 11 12 13 14
      . . I . . . . . . . I . I .
      . . E . . . . . . . E . E .


[LNS1]train-4#[LNS1]train-4# show l2tp peer LAC1 tunnel
Loc    Rem    Remote          Rem    Ses   Act   Total   Total
ID     ID     IP Address      Port   Count Ses   Estab   Fail
-----  -----  --------------- ------ ----- ----- ------- -------
23734  23733  1.1.1.1         1701   1     1     1       0


[LNS1]train-4# show l2tp peer LAC1 tunnel 23734 session
Ses  Rem
ID   ID     State          Port/Circuit
----- ----- -----------   --------------------------------------
60847 34635 Established    L2TP LNS 3027653


[LNS1]train-4# show circuit count l2tp LAC1 tunnel 23734 session 60847
Circuit          Packets/Bytes Sent  Packets/Bytes Received
L2TP LNS 3027653       22359               22357
                       1073261             447140

[LNS1]train-4#
[LNS1]train-4# show circuit count l2tp LAC1 tunnel 23734 session 60847 
detail
Circuit: L2TP LNS 3027653, Internal id: 5/2/3027653, Encap: l2tp-lns
Packets                              Bytes
-----------------------------------------------------------------------
Receive          :            22397  Receive           :      447940
Receive/Second   :             0.00  Receive/Second    :        0.00
Transmit         :            22399  Transmit          :      1075181
Transmit/Second  :             0.00  Transmit/Second   :         0.00
IP Multicast Rcv :                0  IP Multicast Rcv  :            0
IP Multicast Tx  :                0  IP Multicast Tx   :            0
Unknown Encaps   :                0  Unknown Encaps    :            0
Down Drops       :                0  Down Drops        :            0
Unreach Drops    :                0  Unreach Drops     :            0
Adj Drops        :                0  Adj Drops         :            0
WRED Drops Total :                0  WRED Drops Total  :            0
Tail Drops Total :                0  Tail Drops Total

8.10   Step 9: Displaying LNS L2TP Summary Information

On the LNS, use the show l2tp summary command to display information about all peers in all the contexts.

[LNS1]Redback#show l2tp summary
Context Name    Peer Name   Local Name    Tun      Ses
                                          Count    Count
-------------   ---------   -----------   -----    ----
LNS1            LAC1        LNS1          1        0
LAC1            LNS1        LAC1          1        0    

8.11   Step 10: Checking LNS Bindings

On the LNS, use the show bindings command to verify your bindings. For information about this command, see Section 2.7.

8.12   Step 11: Checking LNS Interfaces

On the LNS, use the show ip interface brief command to verify that your interfaces are up.

[LNS1]Redback#show ip interface brief
Mon Jun 27 06:38:05 2009
Name    Address               MTU    State    Bindings
2/1     4.2.13.4/16           1500   Up       ethernet 2/1
2/4     10.13.49.166/24       1500   Up       ethernet 2/4

Recommended Action: If the interfaces are down, see Verifying Interfaces and Checking Port Performance.

8.13   Step 12: Displaying LNS PPP Summary Information

On the LNS, use the show ppp summary command to display summary information for all PPP sessions in the current context. For information about this command, see Displaying PPP Summary Information.

8.14   Step 13: Checking LNS IP Routes

On the LNS, use the show ip route summary command to display summary information for all IP routes.

8.15   Step 14: Displaying LNS Subscriber Routes

On the LNS, use the show ip route subscriber command to check where the subscriber is being terminated. Make sure that the correct routes are installed in the routing table during IPCP negotiation.

[LNS1]Redback#show ip route subscriber
Codes:C-connected,S - static,S dv - dvsr, R - RIP, e B - EBGP,i B
- IBGP
    A,H - derived hidden
    O  - OSPF, IA - OSPF inter area, N1 - OSPF NSSA external type 1
    N2 - OSPF NSSA external type 2,  E1 - OSPF external type 1
    E2 - OSPF external type 2
    I  - IS-IS, L1 - IS-IS level-1,  L2  - IS-IS level-2
    IPH - IP Host, SUB A - Subscriber address, SUB S - Subscriber
 static
       A - Derived Default
       >   - Active Route
Type    Network       Next Hop  Dist  Metric   UpTime      Interface
> SUB A 20.1.1.1/32              15      0     00:01:08    LNS1
> SUB A 20.1.1.2/32              15      0     00:01:08    LNS1
> SUB A 30.1.1.0/32              15      0     00:01:08    LNS1

8.16   Step 15: Verifying LNS Routes

On the LNS, use the show ip route command to display information about all IP routes or routes for only the specified IP address or IP prefix.

8.17   Step 16: Checking the LNS L2TP Process

Use the show process l2tp command to verify that the L2TP process is running on the LNS. For detailed information, use the detail keyword. If the process is not running, use the show crashfiles command to check if there is a core dump. If one exists, contact your local technical support representative.

[LNS1]Redback#show process l2tp

NAME PID SPAWN MEMORY TIME         %CPU  STATE UP/DOWN
l2tp 166  3    3436K  00:00:00.59  0.00% run   00:02:20

8.18   Step 17: Checking the LNS Configuration

On the LNS, use the show configuration l2tp command to check for common configuration mismatches. Use the following table as a guide to checking LNS configuration mismatches.

[LNS1]Redback#show configuration l2tp

8.19   Step 17: Checking the RADIUS Server

If you have configured a RADIUS server, make sure the RADIUS attributes are configured correctly for your clients. For information about checking the RADIUS Server, see Section 16, Troubleshooting the RADIUS Server.

8.20   Step 18: Debugging the LNS

For information about debug the LNS, see Using L2TP Debug Commands.

9   L2TP Debug Commands

Use the debug l2tp command to enable the generation of debug messages for L2TP-related events.

Use the boot active or boot standby construct to enable debug messages during a system reload for the active or standby controller card, respectively.

Use the switchover keyword to enable debug messages while the system is switching from the active to the standby controller card.

Note:  

The SmartEdge 100 router does not support the standby and switchover keywords.

To store debug messages in the system log buffer, use the logging debug command (in global configuration mode). Use the show log command (in exec mode) to display these stored debug messages.

To display messages in real time, use the logging console command (in context configuration mode) if you are connected to the system through the console port. Or, use the terminal monitor command (in exec mode) if you are connected to the system through a Telnet or Secure Shell (SSH) session.

Note:  

For more information about logging commands and the terminal monitor command, see the Command List.

Issue the no form of this command to disable the generation of debug messages for L2TP-related events.

The following example show how to enable the generation of all PPP debug messages for all circuits on port 1 on the traffic card in slot 3 on the LAC1 context:

[LAC1]Redback#debug circuit handle 14/1:1023:63/2/2/1
[LAC1]Redback#debug circuit ppp packet

The following example show how to enable the generation of all L2TP debug messages on circuit handle 14/1:1023:63/2/2/1 on the LNS1 context:

[LNS1]Redback#debug circuit handle 14/1:1023:63/2/2/1
[LNS1]Redback#debug circuit l2tp all
[LNS1]Redback#show debug circuit  //Shows circuits 
Circuit debugging:                //being debugged.
14/1:1023:63/2/2/1

Note:  

To find a circuit handle, use the show circuit circuit-type detail | grep [ip-address | username | mac-address] command. When you use this command, make sure the encapsulation type is correct.

10   Overview of DHCP

10.1   About DHCP

DHCP dynamically configures IP address information for subscriber hosts. The SmartEdge OS provides three types of DHCP support:

• DHCP relay server—The SmartEdge router acts as an intermediary between an external DHCP server and the subscriber (client). The router relays requests from the subscriber to the DHCP server and relays the server responses back to the subscriber.
• DHCP proxy server—The SmartEdge router provides responses directly to subscriber requests. Each subscriber sees the router as the DHCP server, and as such, sends all DHCP requests, including IP address release and renewal, to the router, which then proxies the information to the external DHCP server. The proxy feature enables the router to maintain IP address leases.
• Internal DHCP server—The SmartEdge router provides the functions of the DHCP server; no external DHCP server is required.

11   Troubleshooting Subscriber Connectivity on the Proxy or Relay

Use Table 30 as a guide to troubleshooting specific DHCP relay or proxy issues—that is, to determine where the packets are dropping.

11.1   Troubleshooting Specific CLIPS Issues

When you troubleshooting specific CLIPS issues, check the following areas:

1. The relay or proxy (SmartEdge router)
2. The external DHCP server
3. The subscriber system

11.1.1   Checking the DHCP Relay or Proxy

On the SmartEdge router acting as a DHCP relay or proxy:

1. Check if overall packets are being sent and received by using the show port counters slot/port command.
2. Make sure that the IP interfaces are up by using the show ip interface command.
3. Check if the relevant CLIPS session is sending and receiving packets by using the show circuit counters slot/port command. This command displays the parent and child circuit counters.
4. Check if the DHCP relay or proxy is receiving DHCP discovers by using the show dhcp relay stats command. Make sure the DHCP discover counters are increasing.

   If the results are as expected, make sure the DHCP relay or proxy is relaying discovery messages to the external DHCP server. Look at the sent counters field to see if they are increasing. If the sent counter is not increasing, check if authentication is successful.

5. Ping the external DHCP server to see if it is reachable.
6. Ping the IP address of a remote IP host. If this fails, check the route table entries by using the show ip route command.
7. Check the configuration by using the show configuration command.
8. Examine the ARP entries by using the show arp command.

11.1.2   Checking the External DHCP Server

On the external DHCP server:

1. Check that it has available IP addresses to assign to the subscribers.
2. Verify that the external DHCP server responds to the subscriber request and that the subscriber DHCP client receives this response.

11.1.3   Checking the Subscriber System

1. On the subscriber system, check the IP configuration and verify the correct values for your host.
2. Check if the subscriber system has a static address that is outside the pool range of the external DHCP server.

11.2   DHCP Relay Is Not Receiving Offers

If the SmartEdge router acting as a DHCP relay is not receiving offers:

1. Check if the DHCP relay is dropping discover packets in the TX error counter by using the show dhcp relay stats detail command.
2. Validate the server status by using the show dhcp relay server command.
3. Validate the network configuration by using the show ip route command.
4. Determine if the DHCP relay is selecting the correct multibind interface for using gi_addr.

   If the subscriber is bound to an interface, issue the show subscribers active command to get information about the interface it is bound to.

5. Check the external DHCP server. Make sure the network range is configured correctly.