The dynamic link library (DLL) that you are calling will determine which calling convention to use. The DLL may execute with either convention; however, you may get an error or crash after the DLL executes if the wrong convention is chosen.
The calling convention, in general, refers to how parameters are handled when using a subroutine. When using the C calling convention, the caller is responsible for cleaning up the stack. In using the standard calling convention, the called function is responsible for cleaning up the stack. If the caller (LabVIEW) and the called DLL function don't use the same calling convention, then they will either both take things off the stack or neither of them will. Either situation can cause LabVIEW to crash when the called function returns. The calling convention is specified in the lower right portion of the Call Library Function node window.