Solution
The issue comes from the windows file explorer File name extensions settings, when it is not checked the File Explorer won't show extensions and LabWindows/CVI will not be able to show the list of extensions. In order to overcome this issue check the File name extensions box from File Explorer > View > File name extensions, or instead of
FileSelectPopupEx (directory, "*.xml", "datafile",
"This is a File Select Popup",
VAL_SAVE_BUTTON, 0, 0, filename)
, use
FileSelectPopupEx (directory, "xml file (*.xml);", "datafile",
"This is a File Select Popup",
VAL_SAVE_BUTTON, 0, 0, filename)