Liquid chromatography tandem mass spectrometry is becoming a routine and powerful technique for compound identification in lipidomics and metabolomics studies. However, identifying compounds from millions of mass spectra is critical for data analysis. We built and extended a comprehensive and high quality reference tandem mass spectral library and applied it to fast and accurate metabolite identification in human samples. For building the library, tandem mass spectra of each authentic compound were acquired with Orbitrap Elite and Fusion Lumos instruments (IT, FT-IT, and HCD) at different collision energies. A consensus spectrum was generated from multiple spectra at the same instrument condition for the library. The product ions in each spectrum were annotated with MS_Sing and MS Interpreter software programs (free download at chemdata.nist.gov) and validated by manual inspection. Currently, the library contains >1 million spectra from >25,000 compounds. Of these, ~5,000 are human metabolites and ~1,000 are lipids including phospholipids, glycolipids, steroids and related compounds. Human plasma, urine, and milk samples were analyzed on reverse phase LC/MS/MS (Orbitrap Fusion Lumos) in positive and negative modes respectively. Metabolites were identified by searching the library with MS Search program. Over 700 human metabolites were identified with positive and negative precursor ions (e.g. [M+H]+, [M+Na]+, [M+2H]2+, [M-H]-). The ions produced in-source during the ionization process from the metabolites by various neutral losses (e.g. ammonia loss from tryptophan) were also identified in about 50% of the identified metabolites. These in-source fragments can help confirm the metabolite identification.