Can you work around it by generating a directory list file on the host when a file arrives or periodically?

Pull down the file and use that for retrieval rather than the NLST.

Another option is to turn off 8.3 file name generation on the Windows server if you don't need it.