We present the results of our systematic search for optically elusive, but intrinsically luminous buried AGNs in nearby (z < 0.3) ultraluminous infrared galaxies (ULIRGs) classified optically as non-Seyferts. To disentangle AGNs and stars, we have performed (1) infrared 2.8-4.1 micron low-resolution (R ~ 100) spectroscopy using Subaru IRCS to estimate the strength of PAH (polycyclic aromatic hydrocarbon) emission and dust absorption features, and (2) 20 micron imaging observations of selected ULIRGs using Subaru COMICS and Gemini TReCS, to constrain the emission surface brightnesses of energy sources. Overall, both methods provide consistent results about the importance of buried AGNs. By combining with AKARI IRC 2.5--5 micron and Spitzer IRS 5-35 micron low-resolution spectroscopy, we found that luminous buried AGNs become relative more important with increasing galaxy infrared luminosity (where more stars will be formed). Our results may support the AGN feedback scenario as the origin of galaxy down-sizing phenomenon.