Where the belief comes from
Both are houses you own that are not your main home, and in everyday speech both get called an investment. So people assume the lender sees second home vs investment property as a label they get to pick, usually whichever one seems easier.
Lenders do not see it that way. Occupancy is one of the first facts a file is built on, and the two are separate categories with separate rules.