This paper presents an improved locally optimized short-time fractional Fourier transform (STFRFT), HHT-based locally optimized STFRFT, by finding the optimal order using phase information ignoring the premise of the known chirp rate of signal and pre-estimated pitch of speech. The feature derived from the optimal order FRFT's magnitude spectrum, HHT-based locally optimized STFRFT Mel-frequency cepstral coefficients (HLO-STFRFT-MFCC), reveals the definite advantage in speaker recognition experiments on the TIMIT database. Furthermore, HLO-STFRFT-MFCC yields a gain of 13.0% relative to the baseline feature of Mel-frequency cepstral coefficients (MFCC) in the recognition accuracy on 2004 NIST SRE corpora.