MTU pain with PATS TOF API

For the past few months I've been testing the TOF API against the PATS GT system. By and large the testing has been uneventful except for one persistent problem...

Symptoms

There are two connections to the ASP, one through a leased line and the other over the internet. Since day one there have been issues with the leased line. The TOF would connect and authenticate but then sit idle. What should happen is that all contract data should be downloaded, followed by any orders and fills.

Cause

Eventually we determined that the MTU size was being exceeded. The handshaking process worked because of the small packet size but when the contract download started, the process stopped as the packets were being dropped by the router.

The provider commissioned a Layer 3 MPLS solution. The underlying carrier allowed a maximum MTU size of 1500 bytes but this was reduced by the VPN overhead to 1478 bytes. ICMP was blocked by the firewall so fragmentation was not possible. The end result was that any packets over 1478 bytes were lost.

Solution

Three solutions presented themselves:

• Use a fixed MTU size.
• Enable ICMP between the hosts so a smaller MTU size can be negotiated.
• Change to a Layer 2 Private Leased Line.

In the end, option 1 was implemented for the short term but option 3 being the target long term solution. It will be interesting to see if this eventuates.