How to Transfer Big Amounts of Data Using TCP/IP

Issue Details

I have stored measurement data in an array of 10x50,000 elements. I want to transfer this data from LabVIEW to another computer. As far as I know the TCP/IP protocol caps out at 65,535 bytes. Is there a way to compress the data before sending it over or any way to transfer this huge amount of data in real-time?

Solution

LabVIEW's TCP Write Function and TCP Read Functions are high-level functions that automatically take care of transmitting the data in a way suited for the underlying network connection.

In the example described above the TCP Write Function will split the user data and transfer it using multiple network frames/packets.

Additional Information

Packet Sizes

An IPv4 packet is limited to a size of 65,535 bytes. However the lower layers of a TCP/IP connection typically have lower packet sizes: I.e. the "Maximum Transmission Unit" (MTU) for Ethernet is around 1,500 bytes.

Bandwith consideration

Be aware that the bandwidth of a TCP/IP connection is limited. A 500,000-element array of type DBL like described in the issue above equals approximately 4MB of user data. Transferring this array three times per second already oversaturates a 100MBit/s Fast Ethernet connection.

Example code

This is an example on how to transfer the array from the issue described above using LabVIEW's TCP functions. Execute recipient.vi first, then sender.vi.



This screenshot shows the real network frames being transferred via Ethernet when the above example codes are executed:

Frames #1 to #3 set up the connection, frame #4 is the transmission of the data length (4 bytes), frame #5 is the first one transferring the array. Note that frame #5 and the following ones have a size of 1514 bytes each (except the acknowledge frames like frame #10 sent from the recipient to the sender).