Brocade ADX - The CSW Pseudo Stack
The purpose of this document is to explain the role and functions of the CSW Pseudo Stack.
Within the Brocade ADX feature set is the ability to forward traffic based on layer 7 attributes (such as host header, URI etc.). This is achieved by enabling content switching (CSW).
In order for the the ADX to perform Layer 7 content switching (CSW), the content switching engine must hold the entire client request within its buffer before it can making a forwarding decision. As, in some instances the clients request may be split across multiple packets, the ADX must buffer each client request packet in order to construct the entire client request. To achieve this a buffering layer within CSW is used called the CSW Pseudo Stack.
Note : When enabling CSW on a virtual server, the CSW Pseudo Stack is only used if the virtual server port is pass-through only (i.e non-proxy based). Proxy ports include SSL termination, SSL proxy or tcp-proxy enabled.
In some instances a clients request may span multiple packets. Because of this the ADX will acknowledge (ACK) each packet until the complete request is fully received (buffered). The layer 7 forwarding decision is then made. However, due to the CSW Pseudo Stack not keeping a copy of each packet once it has been sent out, should any of the packets fail to reach the server the ADX would be unable to re-transmit the necessary TCP segment(s).
Within 12.4.00c the ability to buffer each client packet until the server had successfully acknowledged delivery was added. Though this feature is not enabled by default, typically the chance of the packet loss between the ADX and server on a LAN based link is relatively low, in comparison with the overhead required to buffer each packet when enabling this feature. To enable this feature the command 'server csw enable-retransmission' is used.
TCP segments sent from the server to the ADX must arrive in sequence. Segments that do not, are slightingly dropped. However from 12.4.00c onwards the ADX supports both packet-drops and out-of-sequence segments. Again, this is achieved by the ADX buffering each packet until the client has successfully acknowledged delivery.
Note : Within 12.4.00h I have seen instances where the Pseudo Stack has not forwarded out of sequence (i.e TCP retransmitted) packets from the server. To resolve this, the Pseudo Stack was bypassed by *enabling the port as proxy based via the command 'port http tcp-proxy'.
* enabling a port as proxy mode adds additional resource overhead to the appliance and should be added with caution.
To confirm that your issue is CSW based, first check if the connection works when going directly to the webserver. If the connection works when going direct create a test virtual server and disable CSW. Test the connection again, but this time to the test virtual server. If the connection works then troubleshooting should be centred towards CSW.
Note : For general CSW troubleshooting steps please see here.
Based on the caveats outlined earlier in this article the following command is useful for highlighting issues around either packet loss or out-of-sequence packets.
SSH@ADX#show server proxy keep-alive
Keep-alive connection statistics:
In seq. packets = 0 Out of seq. packets = 237656
Fwd exp-seq-wrap ret = 0 Drop wrapped old ret = 0
Retransmit packets = 22232074 Unmatched reverse = 0
Fwd wrapped retrans = 0 Drop old retrans = 37158
Invalid data in wait = 0 Tcb in mem error = 0
No tcb on reverse = 0 Non-conn tcb in ack = 0
Invalid tcb on clien = 0 Unexpected resp data = 0
Unexpected server ac = 0 Src-Nat out of port = 0
Based on the issues described previously in this article there are 4 main solutions,
- Should the issue be down to packet loss between the ADX and server, then upgrade to 12.4.00c and enable 'server csw enable-retransmission'.
- Fix the packet loss in question (on either the server or client side, or both).
- Disable and remove CSW from the binding port on the virtual server.
- Change the port from pass-through to proxy by using the command 'port <port> tcp-proxy'. In turning negating the CSW Pseudo Stack. However be aware that this does add additional load to the ADX, and should be added with care.