Segment Routing

Segment routing (SR) is describe in RFC 8402 and offer flexible way to provide traffic engineering based on the source routing. With SR there is no longer needs to maintain a per-application and per-flow state, and traffic can obey the forwarding instruction provided in the packet.

SR relies on some extension to the routing protocols ISIS and OSPF. The SR can support any type of control plane such as distributed, centralized or hybrid.

Each control plane model provide different set of capabilities and how instructions are allocated and signaled. In a distributed scenario, the segments are allocated and signaled by IS-IS or OSPF or BGP and each node individually decides how to proceed with a packet.

In a centralized deployment, the segments are allocated by a SR controller and the SR controller is responsible for computes the source-route policies.

In the hybrid model two model mentioned above can be use to compute the source-routing policies.

How it works

Each segment has an identifier (Segment Identifier) that is distributed throughout the network using new IGP extensions. The extensions are equally applicable to IPv4 and IPv6 control planes. Unlike the case for traditional MPLS networks, routers with SR do not require (LDP) and (RSVP-TE) to provide traffic traffic engineering.

SR can support following forwarding planes:

  • MPLS where an ordered list of segments is represented as a stack of
    labels
  • IPv6 where an ordered list of segments is encoded as an IPv6 header

Each router and each link has an associated SID (Segment Identifier) where SID for each node in the routing domain participating with SR must be globally unique and is assigned  by a network administrator and represent the shortest path to the router. Unlike node SID an adjacency SID is locally significant and represents a specific adjacency. Early implementation of SR did not allow to change or allocate adjacency-SID manually. It has been change recently and adjacency-SID can be define manually from the range 15000-15999. Manual assignment has several benefits over dynamic allocation such as:

  • Support manually provisioned adj-SID that is persistent over reload and restart
  • Support same adj-SID to be provisioned for multiple adjacencies to the same neighbor
  • Support same adj-SID to be provisioned for multiple adjacencies going to different neighbors
  • Multiple manual Adj-SIDs can be configured for a single adjacency.

NOTE: As I am ware only IS-IS supports manual Adj-SID allocation unless something has change recently and OSPF does it as well so please refer to the latest documentation for more information.

With regard to the IGP control plane there are two basic blocks called IGP-Adjacency segment  and IGP-Prefix segment .

In the context of the BGP these segments are called BGP-Peering segment and  BGP-Prefix segment.

Segment Routing has several benefits such as:

  • SDN ready
  • Simplify network deployment as there is no requirement additional protocols apart IGP
  • Scalability as avoid TE tunnels configuration, avoid large amount of LSP for MPLS-TE, avoid large amount of LDP labels.

This is a simple network setup I will use to show how to configure SR and what command to use to verify it. In addition, I will show some packet capture outputs to verify how control-plane and date plane looks like. That setup is based on Cisco IOS-XE.

Control-Plane

This is basic segment-routing configuration for the diagram above. The only difference on each router is the IP address with the index number under segment-routing section, which refers to the Loopback interface.

interface Loopback0
 ip address 150.1.1.1 255.255.255.255

segment-routing mpls

  connected-prefix-sid-map
   address-family ipv4
    150.1.1.1/32 index 201 range 1 
   exit-address-family

router ospf 1
  segment-routing area 0 mpls
  segment-routing prefix-sid-map advertise-local

The index number must be unique on each devices as described early otherwise a router generates similar error message.

%OSPF-4-SR_DUP_SID: OSPF SR Duplicate Node SID index detected, SID 201, prefix 150.1.1.1/32 (R)

As soon as SR is enabled on the router, MPLS application must registered as a client with LSD (Label Switching Database) to allocate labels. SRGB label allocation range by default is between 16000-23999. The first command below shows how to verify the status of the segment routing and the second one how to verify label range allocation on the box.

CSR1#sh segment-routing mpls state 
 Segment Routing MPLS State : ENABLED

CSR1#sh segment-routing mpls gb 
LABEL-MIN  LABEL_MAX  STATE           DEFAULT   
16000      23999      ENABLED         Yes    

In this example OSPF is used for segment routing and if you want to verify OSPF segment routing status, you can run following command ip ospf segment-routing . This command gives you some basic information about relationship between OSPF and SR. Also it shows you what is the forwarding data plane for SR, what is the OSPF area that SR is enable for, what label allocation method is preferred, label range etc.

CSR1#sh ip ospf segment-routing 

            OSPF Router with ID (150.1.1.1) (Process ID 1)

Global segment-routing state: Enabled

Segment Routing enabled:
           Area        Topology name   Forwarding
              0                 Base      MPLS

SR Attributes
    Prefer non-SR (LDP) Labels
    Do not advertise Explicit Null

Local MPLS label block (SRGB):
    Range: 16000 - 23999
    State: Created

Registered with SR App, client handle: 2
  Connected map notifications active (handle 0x0), bitmask 0x1
  Local export map notifications active (handle 0x1), bitmask 0x2
  Active policy map notifications active (handle 0x2), bitmask 0x6
Registered with MPLS, client-id: 100

Bind Retry timer not running
Adj Label Bind Retry timer not running

With regard to the index number it must be unique as mentioned above due to that prefix-SID is advertised as domain-wide unique index. The prefix-SID points to specific label within the SRGB. The label equals prefix-SID index + SRGB base. For example prefix-SID for the CSR3 router is 203 so based on the label allocation formula, the label for 150.1.1.3 should be 16203. Let’s verify this on the CSR1.

CSR1#sh mpls forwarding-table 
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
16         Pop Label  150.1.1.2-Gi1-10.1.12.2   \
                                       0             Gi1        10.1.12.2   
17         Pop Label  150.1.1.3-Gi2-10.1.13.3   \
                                       0             Gi2        10.1.13.3   
16203      Pop Label  150.1.1.3/32     0             Gi2        10.1.13.3   
16204      16204      150.1.1.4/32     0             Gi1        10.1.12.2   
16205      16205      150.1.1.5/32     0             Gi2        10.1.13.3   

As shown above the prefix 150.1.1.3/32 got assigned the label 16203. All other Loopback IP’s got label allocation following the same logic. At this moment I am not going to discuss meaning of other two remaining labels (16 and 17), these two labels are associated with adjacency-SID which will be explain later on.

Now if you know some details about segment-routing let’s look at the packet capture to see how control-plane works for SR with OSPF and how OSPF advertises N-SID and adjacency-SID. In terms of N-SID all necessary information are encoded into LSA-type 10 and as you can see below the prefix 150.1.1.1/32 got allocated SID 201 which is the index number configured for CSR1 router as shown on the diagram.In addition to prefix-SID allocation you can see that Node flag was set to 1 as well, which indicates that this is N-SID. Another information encoded into LSA-type 10 is SRGB SID label, which in this case is equal 16000 as highlighted below.

In terms of adjacency-SID these information are also encoded into LSA-type 10 where you can find information about the link ID and label allocation. Some interesting information are highlighted below for reference.

The above packet capture output shows that label allocation for adjacency-SID advertised by CSR1 to CSR2 is 16. The flag indicates that that label is local significant only. The range for adjacency-SID is different then SRGB and in this example is assigned automatically by the system. That label is used for static MPLS labels allocation. It can be verify by checking mpls forwarding-table on CSR2 router as shown below. Even the CSR1 router advertised adjacency-SID encoded into LSA as the label 16, the CSR2 router shows Pop Label for CSR1 due to that it was received as implicit null label.

CSR2#sh mpls forwarding-table 
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
16         Pop Label  150.1.1.4-Gi2-10.1.24.4   \
                                       0             Gi2        10.1.24.4   
17         Pop Label  150.1.1.1-Gi1-10.1.12.1   \
                                       0             Gi1        10.1.12.1   
16201      Pop Label  150.1.1.1/32     0             Gi1        10.1.12.1   
16203      16203      150.1.1.3/32     0             Gi1        10.1.12.1   
16204      Pop Label  150.1.1.4/32     0             Gi2        10.1.24.4   
16205      16205      150.1.1.5/32     0             Gi2        10.1.24.4   

Also, you can verify OSPF database to confirm what labels are sent from neighbor as shown below. On the CSR2 router following command was run ip ospf database opaque-area adv-router 150.1.1.1 to display all LSA-type 10 received from neighbor 150.1.1.1. The first LSA contains information about the SRGB range. The second LSA contains information about SID index number assign to the prefix 150.1.1.1/32, Flag N which indicates N-SID and also prefix-SID is included. The last LSA shows what is the adjacency-SID and who is the neighbor on that P2P link. Please note that this output is just a snipped of OSPF database and show some information related to SR advertised by CSR1 to CSR2.

CSR2#sh ip ospf database opaque-area adv-router 150.1.1.1 

            OSPF Router with ID (150.1.1.2) (Process ID 1)

                Type-10 Opaque Area Link States (Area 0)

  LS age: 1574
  Options: (No TOS-capability, DC)
  LS Type: Opaque Area Link
  Link State ID: 4.0.0.0
  Opaque Type: 4 (Router Information)
  Opaque ID: 0
  Advertising Router: 150.1.1.1
  LS Seq Number: 80000007
  Checksum: 0xBF17
  Length: 52

    TLV Type: Router Information
    Length: 4
    Capabilities:
      Graceful Restart Helper
      Stub Router Support

    TLV Type: Segment Routing Algorithm
    Length: 1
      Algorithm: IGP metric based SPT

    TLV Type: Segment Routing Range
    Length: 12
      Range Size: 8000

      Sub-TLV Type: SID/Label
      Length: 3
        Label: 16000

  LS age: 1575
  Options: (No TOS-capability, DC)
  LS Type: Opaque Area Link
  Link State ID: 7.0.0.0
  Opaque Type: 7 (Extended Prefix)
  Opaque ID: 0
  Advertising Router: 150.1.1.1
  LS Seq Number: 80000007
  Checksum: 0x732A
  Length: 44

    TLV Type: Extended Prefix
    Length: 20
      Prefix    : 150.1.1.1/32
      AF        : 0
      Route-type: Intra
      Flags     : N-bit

      Sub-TLV Type: Prefix SID
      Length: 8
        Flags : None
        MTID  : 0
        Algo  : IGP metric based SPT
        SID   : 201

  LS age: 1576
  Options: (No TOS-capability, DC)
  LS Type: Opaque Area Link
  Link State ID: 8.0.0.7
  Opaque Type: 8 (Extended Link)
  Opaque ID: 7
  Advertising Router: 150.1.1.1
  LS Seq Number: 80000006
  Checksum: 0x60CF
  Length: 48

    TLV Type: Extended Link
    Length: 24
    Link connected to : another Router (point-to-point)
    (Link ID) Neighboring Router ID: 150.1.1.2

      Sub-TLV Type: Adj SID
      Length : 7
        Flags  : L-Bit, V-bit
        MTID   : 0
        Weight : 0
        Label  : 16

To verify Node-SID you can run following command sh ip ospf segment-routing sid-database which gives you information about index number used by specific prefix and which prefix is localy originated.

CSR2#sh ip ospf segment-routing sid-database 

            OSPF Router with ID (150.1.1.2) (Process ID 1)

OSPF Segment Routing SIDs

Flags: L - local, N - label not programmed,
       M - mapping-server

  SID                Prefix/Mask
--------          ------------------
201               150.1.1.1/32             
202         (L)   150.1.1.2/32             
203               150.1.1.3/32             
204               150.1.1.4/32             
205               150.1.1.5/32             

Data-Plane

When segment routing is enable for an OSPF, it enables MPLS for any interface on which the OSPF topology is active and then MPLS becomes responsible for data-plane forwarding.

Lets look at CSR1 mpls forwarding-table to see how it looks especially focus on the prefix 150.1.1.5. To reach this prefix the label 16205 is used and that label number is consistent through the network. Each node has exactly the same label allocation number for that prefix 150.1.1.5, because that number is allocated based on the prefix-SID + SRGB as describe above. Each node received exactly the same information encoded into LSA-type 10 and based on those information label is generated. Because every router has exactly the same OSPF database topology view that’s why each router allocated the same label 16205 for the prefix 150.1.1.5/32. Below is output from two routers where prefix 150.1.1.5/32 has the same label.

CSR1#sh mpls forwarding-table 
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
16         Pop Label  150.1.1.2-Gi1-10.1.12.2   \
                                       0             Gi1        10.1.12.2   
16202      Pop Label  150.1.1.2/32     0             Gi1        10.1.12.2   
16203      16203      150.1.1.3/32     0             Gi1        10.1.12.2   
16204      16204      150.1.1.4/32     0             Gi1        10.1.12.2   
16205      16205      150.1.1.5/32     0             Gi1        10.1.12.2   

CSR2#sh mpls forwarding-table 150.1.1.5
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
16205      16205      150.1.1.5/32     1670          Gi2        10.1.24.4   
CSR2#

To confirm that, let’s check again OSPF database running following command sh ip ospf database opaque-area adv-router 150.1.1.5 on the CSR1 router. As you can see there are two very important information here SRGB label which is set to 16000 and SID number setup to 205. Based on these information label 16205 is generated and programmed into data-plane for MPLS forwarding towards 150.1.1.5 prefix.

CSR1#sh ip ospf database opaque-area adv-router 150.1.1.5

            OSPF Router with ID (150.1.1.1) (Process ID 1)

                Type-10 Opaque Area Link States (Area 0)

  LS age: 1344
  Options: (No TOS-capability, DC)
  LS Type: Opaque Area Link
  Link State ID: 4.0.0.0
  Opaque Type: 4 (Router Information)
  Opaque ID: 0
  Advertising Router: 150.1.1.5
  LS Seq Number: 80000008
  Checksum: 0xA52C
  Length: 52

    TLV Type: Router Information
    Length: 4
    Capabilities:
      Graceful Restart Helper
      Stub Router Support

    TLV Type: Segment Routing Algorithm
    Length: 1
      Algorithm: IGP metric based SPT

    TLV Type: Segment Routing Range
    Length: 12
      Range Size: 8000

      Sub-TLV Type: SID/Label
      Length: 3
        Label: 16000

  LS age: 1344
  Options: (No TOS-capability, DC)
  LS Type: Opaque Area Link
  Link State ID: 7.0.0.0
  Opaque Type: 7 (Extended Prefix)
  Opaque ID: 0
  Advertising Router: 150.1.1.5
  LS Seq Number: 80000008
  Checksum: 0xF996
  Length: 44

    TLV Type: Extended Prefix
    Length: 20
      Prefix    : 150.1.1.5/32
      AF        : 0
      Route-type: Intra
      Flags     : N-bit

      Sub-TLV Type: Prefix SID
      Length: 8
        Flags : None
        MTID  : 0
        Algo  : IGP metric based SPT
        SID   : 205

This is packet capture output taken for interface Gig1 at CSR4 router and you can see that incoming ICMP packet from CSR2 towards prefix 150.1.1.5 was label as 16205. Every router on the network has exactly the same label for the destination prefix.

For IPv4 with SR there is still requirement to rely on MPLS data-plane forwarding however there is no more prerequisite to run LDP, as all data-plane capabilities and label ranges are advertised using the SR-capabilities sub-TLV inserted into the OSPF LSA-type 10. With SR we can reduce number of protocols needed to reach the same goal as everything is carried by means of single IGP protocol. SR allows us to reduce network complexity by reducing control-plane protocols and simplify troubleshooting.

The next blog series will be L3VPN and SR.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s