Contents

1   Is This an IPv6 Issue?

The first step in troubleshooting subscriber problems is to perform a general triage of the problem.

Use the BRAS Troubleshooting Guide to get information about the following issues and factors that could cause subscriber problems:

• System status
• Subscriber licenses
• Subscriber connection
• Configuration
• Interface connections
• Bindings
• Authentication
• Traffic flow
• Subscriber routes

Depending on the evidence, continue troubleshooting in the problem areas described in Table 1.

For more information about the commands used in this guide, see Command List.

The following focused troubleshooting procedures assume that you have collected data about problematic subscriber sessions by using the show subscriber active sub-name, show subscriber active all, or show ppp all commands. Note the subscriber name and domain, circuit handle, context, IP pool, prefix, IP address, and any other relevant information.

2   Troubleshooting PPP Problems

For the procedures to check general PPP problems on the SmartEdge router, see BRAS Troubleshooting Guide.

To troubleshoot PPP problems with IPv6 and dual-stack subscribers, perform the steps in Table 2.

2.1   PPP Sessions Are Not Coming Up or Are Dropping

To investigate perform the following tasks.

2.1.1   Check Subscriber Licenses

To start investigating dual-stack subscribers that are not coming up, check the subscriber licenses.

1. To check the status of SmartEdge licenses, enter the show subscriber license summary command.
2. To examine IPv6 subscriber license events, enable AAA debugging with the debug aaa all command.

2.1.2   Investigate Subscriber Status

1. To investigate single subscriber sessions, use the show subscriber active command as shown in Step 7 in Section 2.2 in Section 3.2. You can also use the show subscriber log session cct-handle command, or the show subscriber all | grep sub-name command.
2. To investigate multiple subscribers, use the show subscriber summary all command.

2.2   Are Subscribers Enabled for IPv6?

Verify and troubleshoot the IPv6 stack with the following steps:

1. Verify that IPv4 and IPv6 are enabled and up:

[local]Redback#show ppp all 
Fri Mar 19 11:01:34 2010                                
                             LCP     IPCP   IPV6CP  NLCP MPLSCP 
Port/Circuit          Unit  State    State   State  State    State 
---------------------------------------------------------------- 
2/8 vlan-id 32 pppoe 6   0 Opened   Opened   Opened     
Total circuits: 1  up: 1  down: 0

The expected session should be up and LCP, IPCP, and IPv6CP state should be Opened.

2. To verify that IPv6CP is enabled, you can use the show ppp circuit command to ensure that IPCP came up (yes) and IPv4 is enabled (yes).
3. Verify that dual stack is enabled by running the following debug commands for a short time:

   debug circuit ppp authentication

   debug circuit ppp packet

   For example, in the following segment of the output, look for IPv4 state of Enable and IPv6 state of Enable or Disable. LCP ConfRej displays if IPv4 or IPv6 are disabled. If authentication is failing, you see repeated AAA messages.

local]Redback#debug circuit ppp packet 
Mar 9 15:08:57.000: [2/8:1023:63/6/2/3813]: %PPP-7-AUTH: [0] 
Sending authentication request to AAAd
Mar 9 15:08:57.011: [2/8:1023:63/6/2/3813]: %PPP-7-AUTH: [0] Authentication response status: Success
Mar 9 15:08:57.011: [2/8:1023:63/6/2/3813]: %PPP-7-AUTH: [0] Authentication response: IPv4 Enable
Mar 9 15:08:57.011: [2/8:1023:63/6/2/3813]: %PPP-7-AUTH: [0] Authentication response: IPv6 Enable
Mar 9 15:08:57.011: [2/8:1023:63/6/2/3813]: %PPP-7-AUTH: [0] IPv4 address 192.168.14.254 is from pool
Mar 9 15:18:24.641: [2/8:1023:63/6/2/3814]: %PPP-7-AUTH: [0] Authentication response status: Success
Mar 9 15:18:24.642: [2/8:1023:63/6/2/3814]: %PPP-7-AUTH: [0] Authentication response: IPv4 Enable
Mar 9 15:18:24.643: [2/8:1023:63/6/2/3814]: %PPP-7-AUTH: [0] Authentication response: IPv6 Disable

4. Determine which circuit is having problems by running the debug ppp exception command, which provides data about subscriber circuits with exceptions.

   Copy the circuit handle and use it in the show circuit cct-handle detail command. Look for reports that IPCP and IPv6CP came up.

If IPCP or IPv6CP was not enabled, check the configuration of the subscribers in the RADIUS users file or the router configuration.

5. To verify required RADIUS attributes (Framed-IPv6-Prefix and Delegated-IPv6-Prefix), check the RADIUS users file. You can also verify the IPv6-DNS value and IPv6 options in the users file, as in the following RADIUS example:

user1 User-Password := "test" 
Service-Type = Framed-User, 
Framed-Protocol = PPP, 
Framed-IPv6-Prefix := "6002:2:2:33::3/64",
Framed-IPv6-Route = "3000:1:1::1 5001:1:2::3 7", 
Framed-IP-Address = 10.1.0.3, 
Framed-IP-Netmask = 255.255.255.255, 
Delegated-IPv6-Prefix = "6002:2:3:33::4/56”, 
IPv6-DNS = "1=2000::106:a00:20ff:fe99:a998,2=2000::106:a00:20ff:fe99:a995",
Framed-Interface-Id="200:70ff:fe02:102", 
IPv6-Option  ="source-validation=1",
IPv6-Option +="route-tag=22", 
IPv6-Option   += “nd-profile=nd1”

6. If the attributes do not appear in RADIUS, they may assigned through the local router configuration. Check the subscriber configuration in the show configuration command output:

context local
 interface LOOPBACK0 loopback
  ip address 192.168.1.1/16
  ipv6 address 2001:a:b::1/8
!
 interface SUBSCRIBER multibind
  ip unnumbered LOOPBACK0
  ipv6 unnumbered LOOPBACK0
  dhcpv6 server interface
  ip clear-df
  ip address 192.168.1.1/16
     ipv6 address 12ab:1:1:1::1/48
     ip pool 192.168.0.0/16
!
 aaa authentication subscriber local
 aaa accounting subscriber radius
 radius accounting server 10.1.1.1 key redback
!
 radius attribute nas-ip-address interface LOOPBACK0
 radius attribute nas-ipv6-address interface to_traffic
!
 subscriber name test
  password test
  ip address pool
  ipv6 framed-prefix 12ab:1:1:10::/64
  ipv6 delegated-prefix  4ffe:1:1::/64
  ipv6 nd-profile nd1
  ipv6 source-validation

In this output, you can also determine the IPv6 address assigned to the subscriber interface.

7. You can verify the subscriber status using the show subscribers active sub-name@domain command:

[local]Redback#show subscriber active user1@local
        Session state Up
        Circuit   11/10 vlan-id 101:1 pppoe 1
        Internal Circuit   11/10:1023:63/1/2/120
        Interface bound  SUBSCRIBER
        Current port-limit unlimited
        Protocol Stack Dual
   dns primary 1.1.1.1 (applied from sub_default)
        dns secondary 1.1.1.2 (applied from sub_default)
        idle timeout direction in (applied from sub_default)
        timeout absolute 86400 (applied)
        ascend data filter  (applied in)
           ip in drop dstip 217.89.29.250/32
           ip in forward dstip 224.0.0.22/32 srcip 80.156.111.77/32
           ip in drop dstip 224.0.0.0/4
           ip in forward dstip 58.0.0.0/8 srcip 115.0.0.0/8
           ip in forward dstip 57.0.0.0/8 srcip 115.0.0.0/8
           ip in forward dstip 115.0.0.0/8 srcip 58.0.0.0/8
           ip in forward dstip 115.0.0.0/8 srcip 57.0.0.0/8
           ip in drop
        Framed-IPV6-Prefix  3001:1:2:9101::/64 (applied)
        Ipv6-ND-Profile p1 (applied)
        Delegated-IPV6-Prefix 6001:1:2:9101::/64 (applied)
        Ipv6-DNS  primary 1:1::1  (applied from sub_default)
        Ipv6-DNS secondary 1:1:: (applied from sub_default)

In this case, the subscriber has the required Framed-IPv6-Prefix and Delegated-IPv6-Prefix attributes assigned.

8. For an overview of subscriber status, use the show subscriber summary [ipv4 | ipv6] [all] command.

3   Troubleshooting Subscriber Access (AAA)

For basic AAA troubleshooting steps, see BRAS Troubleshooting Guide.

To troubleshoot subscriber access problems in a dual-stack or IPv6 network, perform the steps in Table 3, which assume that you have turned on logging.

3.1   Enable AAA Debugging and Configure AAA Logging

1. If you have not already done so, turn on debugging with the debug aaa all command. After the problem is resolved, use the no form of the command to turn it off.

   If authentication fails for the session, you may also find the debug aaa acct, debug aaa author, or debug aaa authen commands useful.

3.2   Subscriber Has the Wrong Type of IP Address

Troubleshoot the IP address type with the following steps:

If a subscriber has the wrong IP address type:

1. Examine the access events in the AAA debug output logs for errors in prefix assignment.

2. Examine recent AAA events for the specific subscriber with the show subscriber log session cct-handle command:

[local]Redback#show subscriber log session 4/3:1023:63/6/2/11
---------------------------------------------------------
8       IN      Sat Mar 27 18:42:04.694112
        IPC_ENDPOINT = PPPd, MSG_TYPE = SESSION_UP,
---------------------------------------------------------           
        IPC_ENDPOINT = ISM-IF, MSG_TYPE = IF-BIND,
        Username = user2668@local,
---------------------------------------------------------
        IPC_ENDPOINT = ISM-CCT, MSG_TYPE = CCT-GEN-CFG,
        Username = user2668@local,
---------------------------------------------------------
        IPC_ENDPOINT = PPPd, MSG_TYPE = PROTO_IPV6_UP,
        Username = user2668@local,
---------------------------------------------------------
        IPC_ENDPOINT = ISM-CCT, MSG_TYPE = CCT-GEN-CFG,
        Username = user2668@local,
---------------------------------------------------------
        IPC_ENDPOINT = PPPd, MSG_TYPE = SESSION_DOWN, term_ec = 140
           Username = user2668@local,
---------------------------------------------------------
        IPC_ENDPOINT = ISM-IF, MSG_TYPE = IF-UNBIND,
        Username = user2668@local,
---------------------------------------------------------
        IPC_ENDPOINT = ISM-CCT, MSG_TYPE = CCT-GEN-CFG,
        Username = user2668@local,
        CCT_HANDLE = Unknown circuit
---------------------------------------------------------
        IPC_ENDPOINT = ISM-CCT, MSG_TYPE = CCT-SUB-SESS-DOWN-CPLT,
---------------------------------------------------------

Look for PROTO_IPV4_UP or PROTO_IPV6_UP in the output.

3.3   After Reload or Switchover, IPv6 Subscriber Sessions Do Not Come Up

This section assumes that you already ruled out hardware, network, configuration, and RADIUS problems using BRAS Troubleshooting Guide.

1. Examine the subscriber numbers with the show subscriber summary all command and look for clues in the output of the show subscriber log session cct-handle command.

2. If no other remedy is found, you can restart a module or clear a subscriber and bring it back up:

[local]Redback#clear subscriber username sub-name

3. As a last resort, restart the AAA process and collect troubleshooting data for Customer Support with the process coredump aaad command.

   For more information, see the Data Collection Guideline.

4   Troubleshoot IPv6 Address Pool Problems

4.1   Subscriber Does Not Receive Prefix From IPv6 Pool

If subscribers are unable to get IPv6 addresses from a pool, you may see the log message, "Error getting ip address from pool for subscriber".

To investigate the problem, perform the following steps:

1. Investigate the status of prefixes in IPv6 address pools by using one of the show ipv6 pool commands in the following examples:

[CORP]Redback#show ipv6 pool dhcpv6 123-pool

Interface "123-pool":
   2001:a:c:1::/64           2001:a:c:4::/64         0 in-use,
    4 free,   0 reserved
   4001:a:c::/64             4001:a:c:ff::/64        1 in-use,
  255 free,   0 reserved

[CORP]Redback#show ipv6 pool 123-pool

Interface "123-pool":
   2001:a:b:1::/64           2001:a:b:4::/64                          1 in-use,
    3 free,   0 reserved

[CORP]Redback#show ipv6 pool dhcpv6 123-pool thresholds

Interface "123-pool":
   2001:a:c:1::/64           2001:a:c:4::/64        threshold 
percentage falling 50 log 25 log
   4001:a:c::/64             4001:a:c:ff::/64       threshold 
percentage falling 50 log 25 log


[CORP]Redback#show ipv6 pool dhcpv6 thresholds

Interface "123-pool":
   2001:a:c:1::/64           2001:a:c:4::/64        threshold 
percentage falling 50 log 25 log
   4001:a:c::/64             4001:a:c:ff::/64       threshold 
percentage falling 50 log 25 log

To debug shared prefix pools (also known as AAA or ND prefix pools), use the command without the dhcpv6 keyword.

The dhcpv6 keyword indicates that you are requesting information about the dhcpv6 PD prefix pools in this context, and omitting the keyword means that you are requesting information about the shared prefix pools in this context.

You can also see a summary of IPv6 pool information with the show ipv6 pool dhcpv6 summary command.

2. Examine the configuration of the DHCPv6 server host using the show configuration dhcpv6 command; see the output segment in the following segment of the output:

interface 123-pool multibind
  ip address 10.1.1.1/16
  ipv6 address 2001:a:1:2::1/64
  dhcpv6 server interface
  ip pool 10.1.0.0/16
  ipv6 pool dhcpv6 8001:a:b:1::/64 8001:a:b:100::/64 name pd-pool 
  threshold percentage falling 25 log 10 trap

3. To examine the status of current DHCPv6 hosts, use the show dhcpv6 server host command.
4. To see more prefix-related information, use the show dhcpv6 statistics detail command.
5. To check the assignment of prefixes from the IPv6 pool, compare the number of active subscribers in the output of the show subscriber summary ipv6 all command, with the number of IPv6 prefixes that are in use in the output of the show ipv6 pool summary command.

   If all IP prefixes are assigned from the pool, the total number of active subscribers and IP addresses in use in the pool should be the same.

   To determine if the prefixes are from static configuration or from pools, use the show dhcpv6 server host command.

   When you subtract the number of any statically assigned IPv6 prefixes from the total number of IPv6 prefixes in the output of the show subscribers summary ipv6 all command, the resulting number should match the total number of prefixes in use in the show ipv6 pool summary command.

[CORP]Redback#show subscriber summary ipv6 all
-----------------------------------------------------------
Total=16000

Type            Authenticating          Active     Disconnecting
PPP                   0                  0         0
PPPoE                 0                16000       0
DOT1Q                 0                  0         0
CLIPs                 0                  0         0
ATM-B1483             0                  0         0
ATM-R1483             0                  0         0
Mobile-IP             0                  0         0

[CORP]Redback#show ipv6 pool dhcpv6 summary
1          in use, 16000        free, 0      reserved
260        total, 99  available percentage

6. To determine if individual subscribers have been assigned static IPv6 prefixes, use the show subscriber active command with the username@domain or cct-handle arguments.
7. If the number of subscribers does not equal the number of IPv6 addresses in use, and you can't identify a cause, you can restart the AAA process and collect troubleshooting data for Customer Support with the process coredump aaad command.
8. If the numbers are similar, but subscribers are still not receiving IPv6 prefixes, continue with Section 5.

5   Troubleshooting ND Problems

ND subscribers are bound to interfaces indirectly by being assigned a prefix matching the interface. The ND process advertises subscriber prefixes.

If you are experiencing IPv6 forwarding or routing failures, and suspect that ND is the problem, perform the tasks in Table 5.

5.1   ND Neighbor Entry Missing from the Neighbor Cache or a Ping to the Neighbor Fails

To troubleshoot ND neighbors perform the following steps:

1. For nonsubscriber circuits, ND maintains a neighbor cache for all neighboring IPv6 addresses that have been successfully resolved, as well as for its own local IPv6 interface addresses. To display the cache, use the show nd neighbor command:

[local]Redback#show nd neighbor
IPv6 Address        Age  Link-layer Addr    State  Circuit
2:2:2:2::1                 0    00:30:88:23:23:7f  intf   4/11
2:2:2:2::2                 0    00:30:88:13:2e:9e  reach  4/11
16:1:1:1::1                0    00:30:88:03:01:14  intf   to-sierra-lg
16:1:1:1::2                20   00:30:88:15:b8:2e  reach  to-sierra-lg
2009::1                    0    00:30:88:03:01:0e  intf   2/3
2009::2                    66   00:30:88:03:01:10  probe  2/3
fe80::230:88ff:fe03:115    0    00:30:88:03:01:14  intf   to-sierra-lg
fe80::230:88ff:fe15:b82f   36   00:30:88:15:b8:2e  reach  to-sierra-lg
fe80::230:88ff:fe03:10e    0    00:30:88:03:01:0e  intf    2/3
fe80::230:88ff:fe03:110    77   00:30:88:03:01:10  probe   2/3
fe80::230:88ff:fe13:2e9e   50   00:30:88:13:2e:9e  reach   4/11
fe80::230:88ff:fe23:237f   0    00:30:88:23:23:7f  intf    4/11 

Entries in which the State is marked as Intf are for the local IPv6 addresses defined on the SmartEdge router. All other entries are for remote IPv6 addresses learned from ND neighbor nodes.

2. If an expected IPv6 neighbor address is missing from the cache, use the ping command to verify that ND has lost reachability for this address.
3. Enable ND neighbor debugging with the following commands:
   • debug nd lc
   • debug nd interface
   • debug nd rib
   • debug nd packet
4. Ping the missing IPv6 address; for example, for address 2001::1, enter the ping 2001::1 command.
5. Examine the debug logs; they should show the following:
   • The PPA has requested resolution for the address in a cache_miss message.
   • ND sent an NS message over the interface with a matching prefix, requesting resolution from the neighbor.
   • If the resolution succeeds, the debug output includes a received NA message.

   Use this information in the debug logs to answer the following questions:

   • How far did the resolution attempt get before failure occurred?
   • Did the PPA send a cache_miss request?
   • Did the PPA provide the correct circuit that matches the interface with the correct prefix?
   • Did ND send an NS message?
   • Was an NA received? If so, was it valid?
6. To verify that the circuit received in the cache_miss matches the correct interface (matching prefix), use the show nd interface or show nd circuit commands.
7. To view the ND send/receive packet counters on the interface, use the show nd statistics interface if-name command.

   Use the show nd statistics command to see if NS messages are being sent and NA messages are being received. Examine the total number of subscriber circuits in the output. The NS sent counter should increment when a ping attempt is performed, and the NA rcvd counter should also increment if the neighbor is responding to the request.

5.2   Subscribers Are Not Receiving Auto-Configured Addresses Correctly

The AAA and ISM components handle all subscriber-related configuration information upstream of the ND process. AAA keeps the subscriber configuration and passes the details on to the ISM process, which in turn notifies ND (as well as other processes such as RIB, DHCPv6, and PPA) of all subscriber information for a particular circuit. When troubleshooting, verify that these components pass on the relevant subscriber information correctly. The most important subscriber information is the Framed-IPv6-Prefix attribute used by hosts to autoconfigure their addresses.

The scenario in which subscribers do not receive autoconfigured addresses may occur after a system reload or an XCRP switchover.

To troubleshoot, perform the following steps:

1. Verify that the subscriber circuit is in the Up state and bound correctly to its interface, and that the interface is also Up. To display the subscriber prefixes assigned to a subscriber circuit, use the show nd circuit cct-handle command.

2. Verify that the ND protocol is advertising the prefix in the RA message over the subscriber circuit.

   Once it receives the IPv6 host prefix correctly, ND should advertise this prefix in RA messages to the subscriber. If ND is not advertising the prefix, verify that the RA-interval is not disabled. By default, it is enabled for each circuit, so at regular intervals ND retransmits an RA message containing the prefix. If you have disabled the RA interval by setting the value to 0, the messages are not sent.

   IF the interval is not enabled, an RA message (with the prefix) may still be advertised to the subscriber, but only if the subscriber explicitly solicits the router for the prefix, using a Route Solicitation (RS) message. ND immediately responds to an RS with the RA message.

   Verify that the RA interval is enabled with the show configuration nd or show nd profile prof-name command.

3. To verify that the prefix is assigned to the circuit correctly and find the ND profile assigned to the subscriber, use the show nd circuit cct-handle command.
4. Use the show nd statistics interface if-name command to verify whether the prefix has been solicited.

   The RS received pkt and RA sent counters should increment when the circuit comes up. If the interval is enabled, the RA sent counter should increment every time the RA timer elapses.

5. If it appears that an RA message is being sent over the interface but the prefix is not being received, use the following debug commands to display the contents of the RA message:
   • debug nd packet
   • debug nd packet detail
   • debug nd interface

If the RA is sent, a hex dump of the entire RA message should be displayed. Save the output and send it to Customer Support to analyze whether the prefix has been included in the message, and whether it was advertised with autoconfiguration enabled.

6   Troubleshooting DHCPv6-PD Problems

To troubleshoot problems with DHCPv6-PD, perform the steps in Table 6:

6.1   DHCPv6 Client Does Not Receive a Prefix

1. Check the DHCPv6 server host status:

[local]Redback#show dhcpv6 server host
DHCPv6 Server Host Record:
--------------------------
DUID: 00:03:00:01:00:00:65:01:01:02
   IA Type: PD, IA ID: 0, T1: 6000, T2: 9600
   Prefix: 2001:a:1:1::/64
           preferred lifetime: 12000
           valid lifetime: 14000
           TTL: 9908
           expires at Fri Mar 12 01:27:55 2010

The output includes the DUID, Identity Association (IA) type (that prefixes are applied to), the IA ID, the prefix, and options for the server host.

2. If the DHCPv6 server is up, verify that the DHCPv6 server is enabled on an interface, using the show configuration interface command:

[local]Redback#show configuration interface...
interface sub multibind
  ip address 10.1.1.1/16
  ipv6 address 2001:a:1:2::1/64
  dhcpv6 server interface
  ip pool 10.1.0.0/16
  ipv6 pool dhcpv6 8001:a:b:1::/64 8001:a:b:100::/64 name pd-pool threshold percentage falling 25 log 10 trap
...

Verify that the IPv6 addresses for the DHCPv6 server interface and the multibind subscriber interface match in the configuration. If they do not match, prefixes are not delegated to clients.

3. Examine the DHCPv6-pool prefix assignment using the show ipv6 pool dhcpv6 pool-name command:

[CORP]Redback#show ipv6 pool dhcpv6 123-pool

Interface "123-pool":
   2001:a:c:1::/64           2001:a:c:4::/64         0 in-use,
    4 free,   0 reserved
   4001:a:c::/64             4001:a:c:ff::/64        1 in-use,
  255 free,   0 reserved

4. Turn on debug with the debug dhcpv6 all command.

[local]Redback# debug dhcpv6 all
Mar 12 10:47:09: [0001]: [9/3:1023:63/6/2/2]: %DHCPV6-7-OPT: received Solicit 
from fe80::200:65ff:fe01:102
Mar 12 10:47:09: %DHCPV6-7-PKT: Packet process handler: Process packet 

   0    01 01 00 00 00 08 00 02 00 00 00 01 00 0a 00 03
  16    00 01 00 00 65 01 01 02 00 06 00 02 00 19 00 19
  32    00 0c 00 00 00 00 00 00 00 00 00 00 00 00

Mar 12 10:47:09: [0001]: [9/3:1023:63/6/2/2]: %DHCPV6-7-PKT: XID: 1010000
Mar 12 10:47:09: %DHCPV6-7-OPT: get DHCP option elapsed time, len 2
Mar 12 10:47:09: %DHCPV6-7-OPT: elapsed time: 0
Mar 12 10:47:09: %DHCPV6-7-OPT: get DHCP option client ID, len 10
Mar 12 10:47:09: %DHCPV6-7-OPT: client DUID: 00:03:00:01:00:00:65:01:01:02
Mar 12 10:47:09: %DHCPV6-7-OPT: get DHCP option option request, len 2
Mar 12 10:47:09: %DHCPV6-7-OPT: requested option: IA_PD
Mar 12 10:47:09: %DHCPV6-7-OPT: get DHCP option IA_PD, len 12
Mar 12 10:47:09: %DHCPV6-7-OPT: IA_PD: ID=0, T1=0, T2=0
Mar 12 10:47:09: [0001]: [9/3:1023:63/6/2/2]: %DHCPV6-7-PKT: client ID 00:03:00:01:00:00:65:01:01:02
Mar 12 10:47:09: [0001]: [9/3:1023:63/6/2/2]: %DHCPV6-7-PKT: response XID: 2010000
Mar 12 10:47:09: %DHCPV6-7-PKT: set client ID (len 10)
Mar 12 10:47:09: %DHCPV6-7-OPT: client DUID: 00:03:00:01:00:00:65:01:01:02
Mar 12 10:47:09: %DHCPV6-7-PKT: set server ID (len 14)
Mar 12 10:47:09: %DHCPV6-7-OPT: server DUID: 00:01:00:01:13:2c:1c:c8:00:30:88:00:1c:61
Mar 12 10:47:09: %DHCPV6-7-PKT: set status code (len 2)
Mar 12 10:47:09: %DHCPV6-7-OPT: status code: no prefixes
Mar 12 10:47:09: [0001]: [9/3:1023:63/6/2/2]: %DHCPV6-7-PKT: 
Response source addr: fe80::230:88ff:fe00:1c61, 
destination addr: fe80::200:65ff:fe01:102
Mar 12 10:47:09: [0001]: [9/3:1023:63/6/2/2]: %DHCPV6-7-PKT:
 Sent server response to client: 
00:03:00:01:00:00:65:01:01:02 success.

   0    02 01 00 00 00 01 00 0a 00 03 00 01 00 00 65 01
  16    01 02 00 02 00 0e 00 01 00 01 13 2c 1c c8 00 30
  32    88 00 1c 61 00 0d 00 02 00 06

5. Show the output using the show dhcpv6 log command.

[local]Redback#show dhcpv6 log                                
Time   Evnt SubEvent         Key (ccth|ipv6addr|etc)   Details  
------ ---- ---------------- -------------------------
347.42 Pkt  Pkt from Client  9/3:1023:63/6/2/3         Solicit
                    (cont'd) fe80::200:65ff:fe01:102  
                    (cont'd) 00:03:00:01:00:00:65:01:01:02
347.42 Pkt  Pkt to Client    9/3:1023:63/6/2/3         Advertise
                    (cont'd) fe80::230:88ff:fe00:1c61 
                    (cont'd) 00:03:00:01:00:00:65:01:01:02
347.43 Pkt  Pkt from Client  9/3:1023:63/6/2/3         Request
                    (cont'd) fe80::200:65ff:fe01:102  
                    (cont'd) 00:03:00:01:00:00:65:01:01:02
347.43 Int  IPv6 Add         9/3:1023:63/6/2/3     
                    (cont'd) 2001:a:1:1::/64          
                    (cont'd) 00:03:00:01:00:00:65:01:01:02
347.43 Int  Route Add        9/3:1023:63/6/2/3
                    (cont'd) 2001:a:1:1::/64          
                    (cont'd) 00:03:00:01:00:00:65:01:01:02
347.43 Tmr  Timer Start      9/3:1023:63/6/2/3        
                    (cont'd) 00:03:00:01:00:00:65:01:01:02
347.43 Hlpr Add              9/3:1023:63/6/2/3          
                    (cont'd) 00:03:00:01:00:00:65:01:01:02
347.43 Int  Bind Add         9/3:1023:63/6/2/3             
                    (cont'd) 00:03:00:01:00:00:65:01:01:02
347.43 Pkt  Pkt to Client    9/3:1023:63/6/2/3         Reply
                    (cont'd) fe80::230:88ff:fe00:1c61 
                    (cont'd) 00:03:00:01:00:00:65:01:01:02
347.45 Hlpr Add-Resp         2001:a:1:1::/64               
                    (cont'd) 00:03:00:01:00:00:65:01:01:02
347.45 ISM  R:Cct-Cfg        9/3:1023:63/6/2/3         CCT ethcfg 

6. Verify that packets are reaching the DHCPv6 server, using the show dhcpv6 statistics detail command to display packet statistics.

[local]Redback#show dhcpv6 statistics 
Current time: Fri Mar 26 08:51:14 2010
Last cleared: Never

PKT---------------------------------------------------
 Packets Rx      : 150    Packets Tx     : 150     
 Solicit         : 2      Advertise      : 2       
 Request         : 2      Confirm        : 0       
 Renew           : 146    Rebind         : 0       
 Reply           : 148    Release        : 0       
 Decline         : 0      Reconfigure    : 0       
 Relay Fwd       : 0      Relay Reply    : 0       
 Info Req        : 0      Unknown Pkt    : 0       

Dropped pkt-------------------------------------------
 Solicit          : 0     Advertise      : 0       
 Request          : 0     Confirm        : 0       
 Renew            : 0     Rebind         : 0       
 Reply            : 0     Release        : 0       
 Decline          : 0     Reconfigure    : 0       
 Inform Req       : 0     Unknown Pkt    : 0       

7. Save the output in case you need to send it to Customer Support. Turn off debugging and logging.

6.2   Traffic Is Not Flowing Toward a Subscriber

1. Check the DHCPv6 server host using the show dhcpv6 server host [circuit cct-handle | duid hex-string | prefix host-prefix | subnet prefix/length] [detail | summary] command, as in the following example, which displays the status for a subscriber circuit:

[local]Redback#show dhcpv6 server host circuit 9/3 pppoe 1
DHCPv6 Server Host Record:
--------------------------
DUID: 00:03:00:01:00:00:65:01:01:02
   IA Type: PD, IA ID: 0, T1: 6000, T2: 9600
   Prefix: 2001:a:1:1::/64
           preferred lifetime: 12000
           valid lifetime: 14000
           TTL: 8677
           expires at Fri Mar 12 01:27:55 2010

To determine a host DUID, use the show dhcpv6 server duid command.

2. Verify that the client received the prefix and the route was installed by entering the show ipv6 route command.

6.3   Prefixes Are Not Restored after Reload or Switchover

1. Check DHCPv6 server statistics with the show dhcpv6 statistics command; for sample output, see Step 6 in Section 6.1 in Section 6.1.
2. If the output indicates that the DHCPv6 prefix data is corrupt, you can restart the DHCPv6 process using the process restart dhcpv6d command.

7   Troubleshooting IPv6 Routes

To troubleshoot IPv6 routes in a dual-stack network, perform the steps in Table 7:

7.1   Ping Fails to Reach an IPv6 Circuit

To investigate an IPv6 route failure, start by collecting information about the subscriber circuit.

1. To identify the subscriber address being tracked or debugged, use the show subscriber address username user@domain command.
2. Using the subscriber IPv6 address, enter the show ipv6 route ipv6-address/mask detail command, as in the following example:

[local]Redback#show ipv6 route 2001:a:b:1::/64  detail

    Best match Routing entry for 2001:a:b:1::/64 is 2001:a:b:1::/64 , version 41
    Route Uptime 01:45:48
    Paths: total 1, best path count 1

    Route has been downloaded to following slots
     04/0, 06/0, 11/0 

    Path information :

      Active path :
      Known via subscriber nd , distance 15, metric 0,
      Tag 0, NH-ID 0x34300003, Adj ID: 0xA00000A, Interface sub
      Circuit 11/5:1023:63/6/2/5

Use this command to determine the slots to which the route has been downloaded and the next-hop ID (NH-ID) and adjacency ID (Adj ID).

7.2   IPv6 Subscriber Traffic Is Not Reaching a Destination or Routes Are Missing from the RIB

Using the information collected in Section 7.1, continue troubleshooting the route with the following steps:

1. Enable IPv6 route debugging with the debug ipv6 routing [fib-addition | fib-deletion | redist-addition | redist-deletion | route-addition | route-deletion] command.

2. To investigate a specific IPv6 next hop using the NH-ID, enter the show ipv6 route next-hop NH-ID detail command as in the following example:

[local]Redback#show ipv6 route next-hop 0x34300003 detail
    ** = Via interface
    Next Hop Tbl Version :         41
    Current Next Hops    :          4

NH-ID                 Ref Cnt NH-IP            Via-NH     Interface

0x34300003                3/0                             sub
Adj-id         : 0xA00000A
Mac Address    : 00:1f:3f:3b:7a:5e
Info-Version   : 38                  Node-Version   : 38
Fib Card bits  : 0x4000468           Nh Client bits : 0x0
Info flags     : 0x1                 Lsp ifgrid     : 0x0
IF-GRID        : 0x10000001


Next-hop has been downloaded to following slots
 04/0, 06/0,11/0, 11/1 

In the output, 04/0 is the ingress Packet Processing ASIC (IPPA) of slot 4 and 11/1 is the egress PPA (EPPA) of slot 11.

To look up the next hop by IPv6 address, use the optional ipv6-addr argument (fully qualified IPv6 address) instead of the next-hop ID.

3. To check ND detail, use the show nd circuit and show nd circuit cct-handle detail commands.

[local]Redback#show nd circuit
ND circuit           Grid       Mac Address        Status   Encap Type
11/5:1023:63/6/2/5   0x10000001 00:30:88:03:03:ae  Up       eth pppoe ppp

[local]Redback#show nd circuit 11/5:1023:63/6/2/5 detail
Circuit handle    : 11/5:1023:63/6/2/5   Intf grid      : 0x10000001
Mac addr          : 00:30:88:03:03:ae    Status         : Up
Encap type        : eth pppoe ppp        Circuit type   : Subscriber
Interface name    : sub
Circuit MTU       : 1492
Router lifetime   : 1800
ND Profile        : nd1                  IPv6 Proto State: Up
Subscriber Prefix : 2001:a:b:1::/64

To determine the cct-handle argument, use the show ipv6 route next-hop grid detail command; see Step 2.

You can also use the show nd neighbor command to check ND neighbor status; for sample output, see Step 1 in Section 5.1.